–use_sqlite parameter was not even solving my problem. Inserts on the database were really slow, like 10 per second. A nightmare.

OK, ok, it is an SDK, it emulates the server… it is intended to be used to develop your application, speed is not the most important thing… I agree. But if you have to wait 10 minutes to insert 1000 data entries, then you can desperate. If they were like 1 million in 10 minutes, then I would understand, it would be still slow, but I will understand, but 1000 inserts? 10 minutes? are we all crazy?

Anyway, assuming SQLite seems to be a good database and works usually great, I though I could get that speed improved. Looking at the SQLite FAQ I found following question:

(19) INSERT is really slow – I can only do few dozen INSERTs per second

Basically it seems SQLite should be fine doing 50,000 inserts per second in a normal computer, BUT to guarantee the data integrity it locks inserts until it makes sure they have been written to disk, so if your computer hangs your data remains safe.

These are good news because I don’t care of my local data in the local develoment environment, so in the event of a crash I can just rebuild it in a minute, instead of rebuild it in an hour.

Finally the solution is to use “PRAGMA synchronous=OFF“, and the only way of using this pragma is to edit a line in de Google App Engine SDK, after initializing SQLite database

So you’ll have to edit following file:

google_appengine/google/appengine/datastore/datastore_sqlite_stub.py

Look for the “__init__” constructor and after the connection to the database gets initialized “self.__connection = sqlite3.connect [...]” you can place following statment:

self.__connection.execute(‘PRAGMA synchronous=OFF’)

Changing that line I went from about 10 inserts/second to more than 100 inserts/second. More than a 10x gain for changing a single line. Great!

Now I can continue on the thing I was working on without getting crazy.

With the call above we have 32 hexadecimal characters string (16 random bytes).

Do you know any other approach?

This is called geocoding and after spending half a second looking searching on internet I’ve found two simple REST APIs. Yahoo! and Google provide their own APIs and I’m sure a lot of companies out there make use of it, because of their accuracy.

If you want to use Yahoo! API you need to sign up so they give you some sort of appid and you can start making queries.

In the other hand, Google let’s you do this as easily as:

http://maps.googleapis.com/maps/api/geocode/json?address=Emerson+Street,+Palo+Alto,+california&sensor=false

This URL returns a JSON response which can be easily parsed in all languages (you can also ask for the XML, but I sort of hate XML so I prefer JSON).

Apart from returning the latitude and longitude, it also returns the formatted address, which can be great to normalize the address in your database, in case you need such thing. For example, for the query before, it says the formatted address is:

“Emerson St, Palo Alto, CA, USA”

On top of that, you should always read the Terms Of Service to make sure your app stays aligned with the fair use of the API, but that’s up to you.

Although more inacurate, another interesting project is GeoNames (see the URL below).

For more information:

• Google Geocoding API:
  http://code.google.com/apis/maps/documentation/geocoding/index.html
• Yahoo! Place Finder:
  http://developer.yahoo.com/geo/placefinder/
• GeoNames geographical database:
  http://www.geonames.org/

A typical example of leading zeroes is when you want to show the current time and you want the time to be formatted like hh:mm. There is no problem when it’s 12:40, but when is five minutes past one, you get 1:5, which is not what you would expect. You want it formatted like 01:05. The same happens with dates and in several other situations.

Yesterday I found the best solution I’ve seen for this problem. Before that, let’s see possible solutions.

Usually, needing only one zero, I would do something like:

h = (h < 10) ? ("0" + h) : h;

This is not bad at all, but is not elegant. The problem is, what if we want it to be formatted with 3 digits? Then, using the same schema, we can end up with something like:

h = (h < 100) ? ( (h >= 10) ? ("0" + h) : ("00" + h) ) : h;

Man, this is ugly and unreadable!! Maybe it will improve with an if/else, but it will be bad anyway. Probably if you need to do something like that, you will create a function that will handle this problem and then, you will call that function saying you want h padded with 3 zeros. But there is another solution that does not require you to use or create such a funciton…

Have a look for two digit and three digit:

("0" + h).slice (-2);  // devolverá "01" si h=1; "12" si h=12
("00" + h).slice (-3); // "001" si h=1; "012" si h=12;"123" si h = 123

I don’t know what are you thinking after seeing this solution, but I got flashed when I saw it. Just amazing!

Let’s explain what’s going on. The code above only concatenates the “00″ with the number we want to format (e.g: “00″ + 12 = “0012″) and then, we get the last three digits using the slice string function (that’s why you have a -3). Guess what? The last three characters are “012″ which is the string we want.

Of course this solution is not only limited to leading zeroes, you can use it with leading spaces, or whatever character you want.

Sources:

• Padding zero in Javascript
• Little tricks: string padding in Javascript of Eneko Alonso blog (originally seen here)

Update: Milan Adamovsky has created a web page http://jsperf.com/zero-padding where you can check the performance issues of several functions to pad with zeroes, and you can try it directly in your browser

Anyway, we are all grown ups, so you know what you are doing. Now, imagine you need to know this. There are several ways (the more creative you are, the more ways you will find). The first way, and most simple and complete, is by using the php_uname function. The php_uname is similar to the unix command uname, and with this function we can know the  operating system, the version of the OS, the release number, the host name, and the hardware architecture  (i386, amd64, …). It has all that you need

On the other hand, if you don’t want to use this pretty command, you can directly use the PHP_OS constant, which only includes the operating system name, but in most cases will be enough (it is equivalent to call php_uname(’s’)).

Now you can try these two examples:

echo php_uname() . "\n";
echo PHP_OS . "\n";

The funny thing is that probably you would never need to know the OS, so you can skip the whole post

To start with a simple example, let’s say I’m doing a web site related to the motor industry, and as any other good website, I use a database. In that database there are two tables: brand and model. In the table brand there is a row for each car brand (Toyota, Honda, Hyunday, Mercedes, Chrysler, BMW, Audi, Lexus,…), and in the table model there is one row for each car model. For example, some models for Audi will be A6 2.0 TDIe , A5 2.0 TDI, A4, TT …, and the list grows and grows for each brand.

The think is, that in the front-end of my website I want to list the models of a brand in a specific order (the order I defined).

You will think, well, that’s easy, it’s just adding a field called sort_order in the model table, or if you love to have the tables normalized, then you will create an extra table with two foreign keys to model and brand, and then a field called sort_order.

It does not matter which solution you choose, the final concept is that there is a field name sort_order that is set manually on the backend, which defines the order used to list models in the frontend. Let’s see an example of these tables:

This was just an informal introduction, let’s get started!

In order to simplify things even more, imagine we are always talking about Audi (just to name one), and imagine we are working only with 4 models in the database: A3, A4, A5 y TT. From now on, we will forget about brands. So at the end, our table model looks like this:

Now, when I do a SELECT in my website, and I sort using the sort_order, then the models will be listed this way: A3, A4, A5 and finally Audi TT.

The problem:

¿What do I do if I want Audi TT to be the first in the website?

Please note that I want to move Audi TT to the first position, but I don’t want the A3, A4 and A5 change their respective orders; also note I said the first position, but I could have said I want TT to be placed between A3 and A4.

Solution 1: the easy onel

Well, so the first that comes to mind (at least to me) is basically to increment all rows whose sort_order is greater or equal to the position where I want to place the element I want to move, and then update the element I want to move. Let’s see in SQL:

UPDATE model SET sort_order = sort_order + 1 WHERE sort_order >= 1;
UPDATE model SET sort_order = 1 WHERE name = 'TT';

Above you can see two queries: the first one has a linear complexity because it has to update every single row in the table whose sort order is greater than the position (but all rows should be checked); the other query is just to update the row of the TT.

The problem with this solution is the part where ALL rows should be checked and lots of them updated. Then I though of another solution.

Solution 2: the magic of floats

An idea came to my mind: ¿What if instead of integers I use real numbers?

If the field sort_order was a real number, then maybe I could somehow update just the row I want, and leave the rest untouched. Please keep in mind that I use an interface on the backend as well for listing the products and updating the order, so for all elements I know which is the sort_order for the previous and the next elements, as well as for the element itself.

Now, imagine our initial configuration is (A3, A4, A5, TT), and we want to move the TT between A3 and A4. Then I only have to choose a sort_order value between A3 and A4, and magically the TT will be listed between both values. Let’s see the table after choose the middle value:

Basically I’ve done (1.0 + 2.0) / 2 to get the intermediate value between A3 and A4.

Notice you can also do this as many times as you want, and not only the order will always be preserved, but you will “never find” two rows having the same sort_order.

Let’s see another example, imagine now we want the A4 to be placed between A3 and TT, then the sort_order for A4 will be (1.0 + 1.5)/2 = 1.25.

It’s easy, ¿isn’t it? And you just have to update a single row.

Now there are lots of questions. This is so easy and cool, but ¿what happens if you want to place an element in the first position? If we asume the sort_order starts at 1 for the first element, then the previous element should be choosen as 0.00 (and the same rules will work).

¿And what happens if you want to insert the A6 model? As there are 4 elements, the sort_order for A6 will be 5.0 (to avoid colision, just use number of rows + 1).

And the very good question is: “floating point numbers had limitations, ¿didn’t they? ¿what happens if there is an overflow?” Yes man! this is the question!

Making sort_order to overflow is as easy as keep moving all elements to the same position, over and over again. For example, if we move the element with sort order 3, between the elements with sort order 1 and 2, then the order will be 1.0, 1.5, 2.0, then if we move the 2.0 between 1 and 1.5, we will have 1.0, 1.25, 1.5, if we do repeat the last operation we will have 1.0, 1.125, 1.25, and if you repeat this several times you will get lots of decimals.

This gets us to the third solution

Solution 3: the workaround

Solution 3.1: the background process

One solution could be to have a process once a day,week or month to sort all elements and update the sort_order by their index once sorted, that will remove all decimals for all rows. But if you are going to do this, better implement the first solution.

Solution 3.2: updating to integer when possible

This solution is the same as the solution 2, but rounding numbers to integers before updating, and checking that the number is between (but not equal) to the previous and the next position where we want to be placed.

Let’s see an example starting with following set of rows:

Then if I move A3, between A4 and A5, using solution two I will do the following: (1.7628 + 3.78)/2 = 2.7714. But, if I round 2.7714 down I get 2, and 2 is still between 1.7628 and 3.78, so instead of updating to 2.7714, it will be updated to 2. This way, if I move TT again between A3 and A5, this time instead of computing (2.7714 + 3.78) / 2, the TT position would be computed as (2 + 3.78) / 2, which contains less decimals.

Just notice that in this solution one only has to check if the number is still between the two numbers around, which are already known, and keeps the performance high. Just an update, as before, and all the computations are made on the client-side

If we assume that the movement of all elements will be almost random, using this way of computing number is less probable (but still possible) that an overflow happens.

By the way, I presented this solution using floating point numbers, but you can use integers (and I don’t mean using 10, 20, 30, and then getting intermediate values).

Conclussion

If you want a robust way, use the first solution; but if you want to update just an element each time, because there will be thousands of elements and thousand of users updating the sort order at the same time, then probably the second solution is better, although there can be problems when dealing with concurrent requests.

Finally although I’ve presented a couple of solutions (with their workarounds), I would be pleased to hear of another solutions or improvements.

As you use AJAX  in your web site,  it is not desirable to make the user reload the page for just viewing an update image (not cool at all).

At this point you do some tests, you try to change the “src” attribute of the img tag, you try to remove the previous img and create a new one… but all your effort is useless. The browser has cached the image the first time that was accessed.

Probably the first idea that comes to your mind is, well, if I cannot reload the image, then I’ll make the server generates a new one (with a different name each time), and I’ll load that image with a different name.

You know this is not the best way of doing things, right?

Fortunately there is another easy solution to this: just add a query string at the end of the image src, that changes each time. That would be enough

To make this clear, if your original image has been loaded like this:

<img  id="ExampleImage"  src="http://server/path/to/image.png" alt="" />

Then you should try to reload the image this way:

<img  id="ExampleImage"  src="http://server/path/to/image.png<strong>?nocache=1</strong>" alt="" />

And then, increment the nocache value on each request.

Using jQuery, the code will look something similar to:

$("ExampleImage").attr ('src', imageSource + "?nocache=" + (g_id++));

I’ve seen forums with this kind of question, so I hope this helps someone