Key-value storage

This is only a test for feasibility of usage of KV store as database. You're advised not to use it yet. (You can damage your Opera installation, like I did) So until I figure a safe way to use this - use at your own risk.

Fiddling around with Widget object, I've discovered a good key-value storage. Which probably wasn't meant to be used like I'm going to do below.

During the usage, I'm pretty sure Opera uses some kind of in-memory hash-table, which is dumped to disk upon closing Opera. Awfully speedy, until... well, see below.

Usage

widget.setPreferenceForKey('bar','foo');  // Set
r.write( widget.preferenceForKey('foo') );  // Read

Tests

Compensation test             1000000 iter  5319148  lambdas/sec   
KV write test [same key]        10000 iter   666666  inserts/sec (+-13%)
KV read test [same key]        100000 iter   800000  reads/sec   (+-16%)
KV write test [different keys]   1000 iter Infinity  inserts/sec (+-Inf%)
KV write test [different keys]  10000 iter   312500  reads/sec   (+-6%)
KV write test [different keys]   3000 iter   200000  inserts/sec (+-4%)
KV write test [different keys]  30000 iter   476190  reads/sec   (+-9%)  
KV write test [different keys]   5000 iter   106382  inserts/sec (+-2%) 
KV write test [different keys]  50000 iter   458715  reads/sec   (+-9%) 
KV write test [different keys]  10000 iter    64102  inserts/sec (+-2%) 
KV write test [different keys] 100000 iter   492610  reads/sec   (+-10%) 
KV JSON write [different keys]   1000 iter     8000  inserts/sec (+-1%)
KV JSON read  [different keys]   1000 iter     1686  reads/sec   (+-1%)

Awfully fast! Probably more than enough for average service. Beware activity requests that opera sends to default "_index", I've had testing harness on that URL, Opera hung under those requests :)

Catastrophe and kind of recovery

When I've inserted 100 000 keys, it took a couple minutes for Opera to delete service, when I asked. When you uninstall service - it dumps contents of k-v storage to:

... Local Settings....Application data...\Opera\Opera\widgets\widgets.dat

It took near forever for Opera to load. So 100 000 values is pretty much ceiling for storage. If I use it - I need to keep number of entries much lower.

The format is XML, humanly-understandable:

<section id="320062543375156901831245426259">
<value id="path to widget data" 
            xml:space="preserve">I:\test_apps\test1</value>
<value id="download_URL" null="yes"/>
<value id="network_access" xml:space="preserve">0</value>
<value id="ui key #0" xml:space="preserve">name</value>
<value id="ui val #0" xml:space="preserve">Test1</value>
<value id="ui key #1" xml:space="preserve">serviceName</value>
<value id="ui val #1" xml:space="preserve">test1</value>
<value id="number of saved ui values" xml:space="preserve">2</value>
<value id="number of saved mountPoints values" xml:space="preserve">0</value>

   ---- CUT HERE --- CUT HERE --- CUT HERE --- CUT HERE --- CUT HERE ---

<value id="key #0" xml:space="preserve">test</value>
<value id="key #1" xml:space="preserve">result</value>
<value id="val #1" xml:space="preserve">{"test":"best"}</value>
....
<value id="val #100002" xml:space="preserve">{test:'best',foo:'bar'}</value>
<value id="key #100003" xml:space="preserve">test100000</value>
<value id="val #100003" xml:space="preserve">{test:'best',foo:'bar'}</value>

   ---- CUT HERE --- CUT HERE --- CUT HERE --- CUT HERE --- CUT HERE ---

<value id="number of saved values" xml:space="preserve">100004</value>
</section>

Talk about being open! :) There's a bunch of interesting stuff.
I'll keep it here for later.

Basically, I used a text editor to delete all KV values and edited this:

<value id="number of saved values" xml:space="preserve">0</value>

Opera restarted OK.

Final death of Opera

I've repeated the test with 10 000 key-value pairs (results of which is above). In process my Opera did something wrong and now it's toast. Filing a bug report.

....Opera went back to being operational after a few minutes of cooling down. Maybe using this KV storage isn't a great idea. :)

Bug report: DSK-256646

Works nicely - each try I do this - Opera dies after being closed. Repeating the test - Opera is toast again. :)

After it got back online it suffers explosive amnesia, even forgot how to recognize gestures. Time to take a break.

Slava V. [about me]

← main page

Last updated