In a word: YES.

Twitter's challenges with scaling have made images like the above notorious among users. Not pictured: Fail Whale.

I understand that Twitter is a free service and that they are "operating at scale" for some value of "scale," but I really think that they should find a way to significantly relax their API throttling, or perhaps even eliminate it altogether. Facebook has found a way to do this, and Twitter can too.

It may not seem like a big deal to those who haven't had to contend with Twitter's API, but anyone who's trying to keep a relatively up-to-date dataset based on Twitter data will quickly run up against the 350 calls/hour limit for signed requests. -- and it's a meager 150 calls/hour for unsigned requests, which are very useful, and even necessary, for certain tasks (like when you want to access publicly-available data about a user who hasn't authorized your app).

Now, I'm not asking Twitter to make a big sacrifice or work any magic here -- I just want them to find some reasonable way to make a more developer-friendly API. Below I have a couple of straightforward/obvious suggestions. To my knowledge they are not doing these things, but I welcome corrections from anyone who knows better.

• Charge app developers to raise their calls-per-hour rate ceiling.
• "Eventual consistency" syncing with API servers: replicate API-accessible user data to a separate server farm where there is no need for rate throttling. Data may be a little stale during peak usage hours when Twitter's own app servers are busy, but at least any 3rd party apps that need to make more than 2.5 API calls/minute won't break. API servers could sync with those of the primary service as capacity allowed. I know I personally would be OK with occasionally having data that was ~30 minutes old in exchange for not having to worry about rate limits.

Am I smoking crack here? Is there something about Twitter's service that makes scaling a particularly thorny problem? I would love honest feedback from anyone who knows.

+1!

Can I +1 your +1?

New Twitter API response in one simple graphic...

I've contended with the Twitter API since December of 2006. Here's what I know:

- Yeah, it's harder that we think, and probably what we think matters isn't what's a priority to them as an org, so "technically feasible" doesn't mean it gets done.

- 3rd parties are not relevant to their perceived path to profitability, so it doesn't really matter what you want.

So, yes they could? Will they? Unlikely. Not worth our investment of time, energy and emotion to worry about.

I know that Twitter has suffered through lots of growing pains in the last five years.

Is it possible they're now ready to consider how to get this right?

"You can still use Twitter to grow a business." [email protected]