Allow retrieving a list of URLs #73

New Issue

lucidiot · 2021-03-19T11:04:05Z

lucidiot commented

2021-03-19 11:04:05 +00:00

Some feeds, ~~such as #72,~~ would require a pipeline in which a page is retrieved, some URLs are extracted from the page, then those URLs are retrieved and processed into one RSS feed. Multiple other agencies could benefit from this, as many sites organize their investigations by year.

Some feeds, ~~such as #72,~~ would require a pipeline in which a page is retrieved, some URLs are extracted from the page, then those URLs are retrieved and processed into one RSS feed. Multiple other agencies could benefit from this, as many sites organize their investigations by year.

lucidiot added the

enhancement

label 2021-03-19 11:04:05 +00:00

lucidiot added a new dependency 2021-03-19 11:04:30 +00:00

#72 DICR custom feed

lucidiot removed a dependency 2022-11-19 07:37:28 +00:00

#72 DICR custom feed

lucidiot referenced this issue

2023-10-12 20:23:13 +00:00

Use oq instead of xmltodict #90

lucidiot commented

2023-10-12 20:29:49 +00:00

Xidel supports this: xidel 'https://tilde.town/~lucidiot/itsb/' --follow '//a[img[@alt="rss"]]/@href' --extract /rss/channel/title retrieves the contents of the <title> element of each RSS 2.0 feed listed on the ITSB homepage.

[Xidel](https://www.videlibri.de/xidel.html) supports this: `xidel 'https://tilde.town/~lucidiot/itsb/' --follow '//a[img[@alt="rss"]]/@href' --extract /rss/channel/title` retrieves the contents of the `<title>` element of each RSS 2.0 feed listed on the ITSB homepage.

lucidiot commented

2024-02-24 19:52:26 +00:00

Basic support for Xidel implemented in d4317056f2

Basic support for Xidel implemented in d4317056f2ccc31c0eba098df39748b60ab89b2f

lucidiot closed this issue

2024-02-24 19:52:26 +00:00

Sign in to join this conversation.