Montag, 30. Mai 2011

Design restructured

The synchable class has been removed, the translation between local an foreign ids is now available in a translation model. With this the module can easily installed without editing the ModelStorage.

The organization of the models to synch has also been changed. When a model is added into "synchronize.model" the model will be searched recursively for dependencies. All dependencies are added as children to the proper parent. For each model the fields that are required are automatically added, additional fields can be added manually. With this unneeded fields wont be fetched to save traffic.

The next steps will be:
  • Add handling for function fields to create_many method
  • Add updating for records (not much work)
  • Thinking about synching models with other systems for migration. (Thats not part of the GSoC but I really would like to have this functionality)
  • Testing if the system works correctly
The repository is now available under http://hg.holtzberg.de/synchronize

Montag, 23. Mai 2011

Speed improvement

After profiling the syching procedure I found out that the bottleneck was not the foreign_id to local id relation but the ModelSql.create method which only allows to create single datasets. I added a create_many method to the Synchable class to create datasets with prepared statements which improved the speed very well. The RAM consumption is now even less than before.

Now I  can synch with another Tryton server over the Internet 2400 product.product datasets with all needed dependencies in kind of exactly one minute.

Now the bottleneck has moved to the fetching of records from the master server, but there are lots of possibilities for further optimizations.

The next days I will clean up the code and push it to a public repository for reviewing.

First try of synching with remote Trytond servers

I implemented a synching system for tryton models for testing.
The communication is based on the normal NET-RPC which is also used by the GTK-client. The advantage over a separate solution is that a "master" server needs not to be modified before another server can synch with it.

My first tests have shown that this way of synching is possible.

The next steps will be:
  • Improve synching speed with some kind of caching
  • Preventing creation of system records with foreign ids like the uom model
  • Possibility of excluding fields from synching