I hit a problem developing with App Engine which took me a long time to figure out. I have been flipping between projects so didn't really notice when GAE and GWT started so run so slowly. I have also been experimenting with JRebel and messing with JVM memory parameters. Well "suddenly" every started running really really slowly and App Engine would take around 10 minutes to start up and run my 6 async parallel queries. Was it the new version of App Engine Java introducing a concurrency problem? Was it that my local datastore local_db.bin was too big - holy shit its 10MB!
After a lot of kafuffle I finally removed all break points and viola! Every thing was lightening fast again! What a massive waste of time.
Actually I have just added a really nifty optimisation to Twig for when you are merging multiple parent queries (relation index entities) which is needed for Target Rooms geo search. It now asks each child query which entities it wants and gets rid of duplicates before sending off a single request for all of them. When any of the children needs more parent entities it asks this EntitySupplier for more which in turn asks all of the children again if they need any more.
It has speed up the geospatial search HUGELY! I only recently realised that the complex part of the query (searching 6 blocks of geospatial data in parallel and sorting it by price) was completing in about 50ms!!! The simple part (bulk get of the parent results) was taking between 200 and 1200ms to do 6 bulk gets. What a surprise. You can actually see in the Google App Engine Status page that some days each simple get (not even bulk get) can take like 500ms.
Now Twig combines all bulk gets for each child query into a single bulk get which has made a huge improvement.
