Jeromakay

Vimeo implemented OAuth support on their end, the announcement was made here.

Seems like the old web authentication methods are already marked as deprecated, however they still work 4 days later.  The deprecated methods are

• vimeo.auth.checkToken
• vimeo.auth.getFrob
• vimeo.auth.getToken

The new OAuth specific URLs are (Twitter alike):

Request Token: http://vimeo.com/oauth/request_token
User Authorization: http://vimeo.com/oauth/authorize
Access Token: http://vimeo.com/oauth/access_token

I noticed a big similarity between Vimeo’s and Flickr’s authentication API, and probably Flickr will do a switch soon as well, there’s no OAuth implementation on their end at the moment.

A thing I noticed is that the Vimeo API now requires the OAuth specific data and valid tokens which kind of forces everyone to switch to the OAuth tokens, instead of the old ones extracted via the deprecated methods.

Loopthing needs an unexpected code update because of this right after launching the authentication feature on our end last week, but I’m happy security has been tightened around the video hosting service.

Update: The OAuth specific URLs for Vimeo does not support SSL because of the improper setup of the  certificate and the weird redirects. In the best case you’ll get a 401 error with the message “Invalid nonce”.

I’ve been playing with the Twitter  API again (authenticated REST requests this time), and kept getting this nasty 417 Expectation Failed HTTP error.

I checked the sent headers, there was no Expect header being sent, until I realised the requests I was making to twitter using POST HTTP method weren’t sending the variables I wanted to POST, or they were incorrectly formatted.

So, if you’re ever playing with Twitter’s API, make sure that when POSTing data, variables follow. This ate 30 minutes of my life.

Now, this happens only for requests where POSTed data is expected.  I guess the response Twitter sends is a general one, usually, the 417 HTTP Error shows up only when there’s an Expect header present.

I haven’t read the RFC specs for what can happen when no data is POSTed, but I guess a less ambiguous message than this, could be taken into account.

Internationalization can be messy when developing applications targeting multiple countries.

If you are developing a DB backed application where users can enter non latin characters, you may want to retrieve those rows which have non-latin characters used by different non-english languages, like: Hebrew, Arabic, Chinese, Thai, Hindi, Swedish, Romanian, etc. (such as: ăâîşţ).

If you want to check all the rows having content which doesn’t use only latin characters, you can try the following query which returns these rows.

SELECT `column` FROM `t` WHERE CHAR_LENGTH(`column`)<>LENGTH(`column`);

In my case, the table.column has an UTF-8 character set.

I’ll let you guess why rows containing non-Latin (non-ASCII) characters are returned by the above query.

Reference: MySQL’s String Functions

If you’re getting an error like “403 Service Over Qps” or “ERR_403_SERVICE_OVER_QPS“, you’re probably playing with an API provided by the Mashery.com webservice.

Some of the sites serving their API content via Mashery are The Guardian, and compete.com. The trouble I have comes with compete.com, however, I’ve seen people dealing with theguardian.co.uk as well.

The reason why you are seeing this is probably because you are sending over too many requests in so little amount of time.  To prevent this from happening, you should check the API’s service limits, and also test it against the real possible limits, e.g. if they say 5 requests max per second, try 4 requests per seconds, the limit may be lower than documented.

This error also comes with a HTTP response code of 403, Forbidden.

This summer is being a blessing for Loopthing, the business networking platform. We just finished integrating the Twitter functionality for business profiles, and there’s few more other features integrated as well.

A sample page where twitter is the news feed page of the Loopthing company profile.

Besides the Twitter integration, we also added RSS feeds news feeds reader which also adds content to the profile page of any company joining Loopthing.

Two more other features were also integrated into the networking functionality of the site. The networks members can now upload media files (audios, videos and images), and they can also store files into the repository of each network.

I’m going to keep you posted on updates related to Loopthing, but for now, this is it! Here’s the official blog post announcing the Twitter integration feature.

Party time for 14.7 seconds

Been thinking on this for a long time, how would you index content of so many sites and figure out what part of the whole source is actually the one people are interested in? I’ve met this challenge few times already, once in the old dead Octora search engine (luckily it was an RSS search engine), and also for Pingoat, while experimenting a new feature two years ago.

Well, if this paper (PDF) approaching the problem thanks to substring amplification was available at that time, things would have been a bit easier. The study has been completed by Daisuke Ikeda, Yasuhiro Yamada, and Sachio Hirokawa and it has been titled Formulation of Template Discovery Problem and an Algorithm using Substring Amplification.

In simple words the paper explains how a template pattern can be discovered thanks to a linear time algorithm.

I’m mentioning this paper because I’m sure there may be others out there trying to figure out how to extract the content from a web page by removing those elements which are specific to page’s template. It also contains an example on how the algorithm can be applied to a set of pages sharing the same layout.

I’ve got my lesson learned today while working on a major MySQL based feature for Loopthing. While everything was planned and coded carefully, it seems that one issue got carried away in production mode where things didn’t go just right obviously.

The end result was a slow or non-responsive application. I never though it could be the database system until checking the system’s usage statistics and realizing that mysqld was using over 190% of the CPU power.

This was crazy and unexplainable but what I’m about to reveal now doesn’t have anything to do with database optimization practices, but rather with design practices.  Here’s what happened:

I am a true believer that premature optimization is the mother of all evil (Donald Knuth), after learning it the hard way. The problem now stands in the opposite of this: not optimizing things sufficiently enough: what I did was to put together all my tables structure, setting up the main indexes, and nothing more.

Built my queries around this basic structure, tested on small amounts of data, but then, forgot to fine tune things and see where potential indexes are needed. This was totally forgotten in development code as well, and everything was one day migrated into production.

When production activity started growing things changed and the ugly face of poor queries optimization and index setting was revealed to us here. Geesh.

Now, if your CPU’s cycling like crazy, it’s time to have a look over your db’s structure once again, maybe you’ve “missed” to consider some very important things you always knew you must not be careful not to miss.

On the other hand, there’s a pro of my issue today: it prevented useless hours of premature optimization and benchmarking, LOL

Just been wondering lately if Odesk or RentACoder are still good places to find talented professionals and my conclusion is that none of are what RentACoder used to be four-five years ago. There is no enthusiasm about these sites from my side.

I’m still trying to figure out the reasons why this happens but obviously there’s a big shortage of affordable, talented developers where you can outsource programming.

I’ve been browsing some other new or already existing sites as well, and problem is the same however, I’ve found that recruiting developers through other medias produces better results.

So far I’ve been using mainly social networking sites to find talented people and this proved to be a big success.

If you’d ask me, I wouldn’t outsource via these sites anymore. In the future I hope that Loopthing will be a good source for talented people as well (thinking in progress here…), however my advice is to pay attention to the social networking sites: they do have talented people, and talented people are proud of their work and they’ll tag themselves using the right keywords so you’ll find them easier.

Let the social networks headhunting start now!

Subversion is a great tool, but sometimes because of the operating system where it runs on, you may get some nasty errors.

A good example could be the one that happened to me recently while I was trying to merge two different branches in the same repository. I’d share this with the hope that will help somebody save some time and avoid unnecessary headaches.

The error I’ve got was referring to a file inside my repository right when I was trying to commit the merged result:

Can’t open file ‘/path/to/file’: Permission denied

After some time trying to figure out what’s wrong, I’ve quit playing with permissions I’ve found the solution: changing the permissions of the whole repository files’ to 777.

Since this is not a shared environment, the settings will do just fine and you’re ready to go.

Ok, I admit: I’m a details freak and I consider this part a vital function of my work. Being careful about details is in my opinion the difference between wrong and right and success and failure. I’m expecting someone to prove me wrong since I’m tired of skipping through websites that don’t even deserve anyone’s attention, still they get it because they’re getting some publicity.

Pic credits: Delhi’s Telecommunication. Fully wired! by Brajeshwar

Sooner or later, if those detials aren’t solved, and there’s nobody to notice them missing, failure comes in. So, what makes details so important?

I’ve compiled a nice list of data, which you can easily browse, print, save, and have a look on it later, whenever you want to improve the way you do things.

Why do it?

• to increase the quality of your work: Quality attracts quantity not the way around, if you want people around you to be happy about your work, show them you care. They’ll notice it in every margin and corner of your final product
• it pays off faster: websites/web applications that are quality built, are getting a higher amount of visitors than the buggy ones. While developers are solving bugs in one app, the others are nicely thinking on what can be improved, or added. Makes the difference and users are happy.

Like any serious thing, you don’t need too many arguments to understand that you need it. This is one of those serious things. Let’s find out how to implement the attention to details.

How to do it?

• where there’s a will, there’s a way: you need to be willing to increase the quality of your work, this by default will turn you into a machine focused on doing things right.
• patience: experience and spotting problems is a skill that builds up in time with practice a proper QA oriented minset all the way there
• focused POV: it requires that you should focus on the design of the feature, thinking from the perspective of the user, not of the developer (better to think how it would be easier for me to use this feature as a regular user, rather than a programmer)
• there’s some more left: always ask yourself, is there anything else that I should consider before saying “I’m done with this for now”? If it is, write it down, work on it later. Have sessions where you prepare next release.
• don’t stumble: do major things as planned, while small bits are written down for later or solved at the end of the day when energy is lower but easier things can be done

Look again on the list above, it’s very important, if you don’t understand ask me via email or by posting a comment, I want to make sure you get this right.

How to get started?

1. Evaluate the situation: look at your current work, see what things are there and they don’t work, be honest about it, write it down
2. Prioritize those things with two option: important (1), not so important (0)
3. Start working on the important issues first, while keeping in mind what you have learned inside this blog post, write those new details down, move on and then go back to step 1) and repeat until you’re sure you have a decent product

The last step can spread over a period of a day, up to few months, depending on how big your project is. This is the only way to making sure you are the best in yoour area of activity. If things wouldn’t go like this, all those engineers from Google or Youtube will have no reason to be employed now.

You are not doing it right if:

• you developed paranoic thinking: extreme care for details is not good either, and it will take you nowhere and it’s going to slow you down
• you end up with one big, unmanageable TODO list

You are doing it right if

• quality improves and you deliver a product that stands out
• you can estimate project duration easier and you always have in mind the “details” but without loosing your focus
• you always have a nice “todo” which you can use for your next release candidate and stakeholders love that

Got to read: http://highscalability.com/youtube-architecture Have a look at the “Lessons Learned”, I like the “Constant iteration on bottlenecks” solution, that shows the cyclic way of the three steps I represented above. Reading the whole article is actually not a very bad idea either.