Refreshing amaGama

I recently started working again on improving the amaGama translation memory server and service. The project provides a translation memory system that is used by translation tools such as Pootle and Virtaal. The web service that the Translate project hosts contains translations of several popular pieces of free and open source software. This provides translators in over a hundred languages with suggestions from previous translation work in FOSS localisation. Several areas of amaGama require work, and I wanted to prioritise well so as to reach a number of goals.

Firstly the server itself didn’t receive the attention it needed in the last while. The service was not responding at all, and a number of updates were necessary. I’ve already upgraded the operating system, but a review of the system configuration was also required. Users of Virtaal will be happy to know that I implemented the necessary changes so that the amaGama plugin in Virtaal is working again. On the server things are working at least as well as before, and better in a few areas.

Performance on the service has been inconsistent for many years. There are a number of reasons, including server configuration, and the code itself. I’ve often seen some requests taking more than ten seconds. A translation memory response arriving that late is unlikely to be useful. By that time I have probably translated the whole segment from scratch and moved on. I believe most users need a response in less than a second. Since network latency alone can take more than that, we really need the web service itself to be as fast as possible.

I hope to write soon about interesting changes in the code to improve performance, but I already improved things with simple configuration changes. While there are certain database queries that are slow, handling other responses at the same time allows other users to be mostly unaffected, thereby reducing the impact of a performance problem in one request. Before the server only served single requests at a time—I have no idea why it was configured like that. Some requests still take more than 10 seconds, but this does not occur as frequently any more. The slow responses deserve a blog post or two of their own, and I’m still working on that. (Update: Since then I spoke about this at PostgresConfZA.)

The current database (the memory of translations) on the server is pretty old by now. I’ve started working on refreshing all of that data as well. That is almost a whole project in itself! Many projects moved their version control systems in the last few years, and in some cases I can’t easily find some of the things we included in the database before. If there are specific projects you think should be included in amaGama, feel free to contact me.

Another goal in all of my work is to invest in making things easier in future. The server configuration is simpler, the configuration of the web service is moved out of the code, etc. Hopefully it means that a small volunteer group (even if it is as small as me) can keep this going for a long while still.