Caching RSS reads

When I read the feed for the worldoutline there is no caching, so when I pushed the link to Twitter, the server had to do a billion times more work than it should have. It had to:

Read the feed.

Convert it to internal format to mask diffs betw RSS, Atom, RDF.

Convert it to an xstruct.

Convert the xstruct to HTML.

So, it's time to add caching. Then I realized I would have to do everything that op.getNetOutline does, because without thread management, it would crash when a feed changed while we were rendering something. It may sound unlikely, but it's not at all unlikely, with a server that's running 24 by 7. Someone might use a worldoutline server mostly to display feeds, esp now that they look so sexy!

Then I wondered how much there was specific to OPML in op.getNetOutline, and guess what, there's only one thing. It set flAcceptOpml to true on the call to tcp.httpClient, that just adds an Accept header saying you'd like to get OPML. What if I added an optional param that said, don't ask for that. Then I bet it would work for any XML-based format.

Plan

xml.rss.getFeedItems takes a new optional param, adrxstruct. If it's non-nil, we skip reading the feed, and just parse the struct that's passed to us.

op.getNetOutline takes a new optional param that suppresses the flAcceptOpml feature.