Making a service that lots of people use is always an interesting thing to do. And sometimes and exciting experience too. This month has been especially exciting for TwitRSS.me one way and another. Here is a quick roundup of some of what has been occuring.
Sponsorship from Bytemark
The biggest news is that the lovely folks at Bytemark have sponsored the project. They are picking up the tab for hosting it on their BigV platform. So far the experience has been remarkably smooth. The cool thing is that, because I no longer need to rely on caching on cloudflare and because I've got some more memory and CPU to play with, I have been able to drop the caching time to just an hour — instead of a day as before.
I have deployed Varnish for doing the caching and Pound which lets you serve pages from your Varnish cache to SSL as Varnish doesn’t
Watching traffic pour in video
Another advantage of having the traffic flow through a caching layer that I control is that I can see the traffic as it arrives. Here is a short video I made illustrating how it looks as traffic pours in to TwitRSS.me.
Using Net::Curl with LWP UserAgent
One very interesting thing that I found out when moving to Bytemark was that using LWP::Net::Protocol::Curl makes the difference between the server running with a load of over 30 and running with a load of less than 1. Word to the wise, if you are doing much work with LWP and you are seeing high CPU usage then switching to LWP::Net::Protocol::Curl. Its dead easy just do this:
Port to node.js
I’ve not yet had a chance to review it properly let me know in the comments if its good.
While we are on the subject of ports, someone is running a Chinese version of TwitRSS.me. Pretty cool.