stuartl/tornado-news
1
0
Fork 0
mirror of https://github.com/sjlongland/tornado-news.git synced 2025-09-13 10:03:14 +10:00
Code Issues Releases Wiki Activity
26 Commits 1 Branch 1 Tag 66 KiB
master
Commit Graph

26 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stuart Longland
4571de0d45
requirements.txt: Add a requirements file for pip 2022-05-09 08:33:57 +10:00
Stuart Longland
c620a12d8e
tornadonews: Explicitly specify YAML loader 
This is needed for newer versions of the `yaml` module.
 2022-05-09 08:26:04 +10:00
Stuart Longland
1b906b760e
README.md: Project now requires Python 3.4. 
Last commit I added `yield from`, which is a Python 3.4 feature.  I had
intended to retain Python 2.7 syntax, and I guess if that's really
wanted, we can change that `yield from` to a `yield`, but right now I'm
thinking: onward and upwards.
 2020-04-26 09:25:19 +10:00
Stuart Longland
d4b0329fda
Wrap Tornado's HTTP client fetch to implement callback. 
Tornado's `fetch` method no longer accepts a callback function, it's
purely coroutine based.

Not quite ready to jump to the "new world order" as this code was
written with Python 2.7 support in mind, so we'll make a coroutine
wrapper that calls Tornado's HTTP client, and manages the callback
logic.
 2020-04-25 09:02:10 +10:00
Stuart Longland
9ee623f6bf
Gracefully handle absent PREFERRED_XML_PARSERS 
Newer `feedparser` does not expose this symbol.
 2020-04-25 08:51:38 +10:00
Stuart Longland
b24ee0bccc
Write RSS code as UTF-8 2019-06-03 20:17:42 +10:00
Stuart Longland
08221046a5
On error, dump the feed we tried to parse 
Try to give more diagnostic information when a feed fails to be parsed.
This can be used to troubleshoot code later.
 2019-06-03 20:17:42 +10:00
Stuart Longland
d1acc132d2
Try to handle different feed styles 
Sometimes feeds provide the entire post as HTML (e.g. Gentoo Planet),
some only give a summary in HTML (ABC News) and some only give a
plain-text summary (ABC News previously).

This tries to support a few different formats gracefully, before giving
up and just displaying "Summary not available".
 2019-06-03 20:17:38 +10:00
tripleee
876cbd1c64 README.md: create PRs in branches 
A small note to prospective contributors.
 2018-09-01 08:21:35 -05:00
tripleee
c02ff557e0 Add pyyaml dependency 
Undeclared dependency
 2018-08-31 04:59:38 -05:00
Stuart Longland
ee9d64736e
Update package metadata for v0.1.0 2017-10-15 08:49:30 +10:00
Stuart Longland
a622f05ab9
Add .gitignore and README.md 2017-10-15 08:45:46 +10:00
Stuart Longland
495c14ccd0
Encode digest data as UTF-8 before digesting. 2017-10-15 08:35:29 +10:00
Stuart Longland
0a7497fba6
Handle entries that have no HTML content. 
Some sites, notably ABC News, provide just plain-text content or a
practically identical HTML summary, resulting in neither appearing on
the feed.  This checks the output of the HTML content extraction and
falls back to the summary if that comes up empty.
 2017-10-15 08:32:18 +10:00
Stuart Longland
9acacf2a96
Improve error reporting on failed fetch. 2017-10-15 08:02:48 +10:00
Stuart Longland
0e0e566756
Allow disabling of If-modified-since 2017-10-15 08:02:48 +10:00
Stuart Longland
da2e25f58e
Add connect_timeout and request_timeout options 2017-10-15 08:02:47 +10:00
Stuart Longland
3b2a520177
Handle missing id on entries. 2017-10-15 08:01:32 +10:00
Stuart Longland
b353f70cfa
Handle feeds with entries that lack IDs 
The Australian Bureau of Meteorology is one such feed.
http://media.bom.gov.au/rss/

Instead, we'll generate one using the SHA1 hash of the link.
 2016-07-14 20:08:20 +10:00
Stuart Longland
e107cee304
example: Brighten the links a bit 2016-03-13 08:26:19 +10:00
Stuart Longland
ccd08db986
example: Update example template and styling. 2016-03-13 08:16:19 +10:00
Stuart Longland
13b581fcc7
Expose the number of workers to the config file. 
We default to the number of CPU cores, but this allows a fixed value to
be configured.
 2016-03-06 17:37:35 +10:00
Stuart Longland
fa4de0bbc3
Permit limiting the number of posts. 2016-03-06 17:06:54 +10:00
Stuart Longland
0d5546abd4
If read fails, fall back to cache. 
We can probably factor this a bit better so we're not repeating
ourselves, but in essence, if we have an older version we can put up
instead, we'll use that.
 2016-03-06 16:54:55 +10:00
Stuart Longland
16ad2d14e1
Fix handling of 'content' 2016-03-06 16:49:20 +10:00
Stuart Longland
10f2c56945
Initial check-in of Tornado News 2016-03-06 14:58:34 +10:00