Documentation about making tests

[FSX]

Documentation about making tests. With some code example ofcourse.

[haizaar]

Frank,
What did mean here exactly?
Writing tests is pretty obvious one you look at tests.py.

[FSX]

Just as the title says. To make writing tests a little bit easier.

[kamelzcs]

Maybe the parallel test is not good enough.

When I run the test, nearly all the parallel tests failed.
1.10 is really not strong enough in self.assertLess(execution_time, sleep_time*1.10, msg="Query execution was too long") .
In my case, the execution_time would always between (sleep_time*1.1, sleep_time*1.3].

Any idea to improve parallel test?

[haizaar]

Can you tell me more about your setup?

It works for me with server located over LAN. It also runs on Travis CI without any issues.

[kamelzcs]

System and pip versions:
Ubuntu 14.04
tornado (4.0.2)
psycopg2 (2.5.4)
psql (PostgreSQL) 9.3.5

Test code:
python setup.py test

Result:

Using psycopg2
running test
running egg_info
writing requirements to Momoko.egg-info/requires.txt
writing Momoko.egg-info/PKG-INFO
writing top-level names to Momoko.egg-info/top_level.txt
writing dependency_links to Momoko.egg-info/dependency_links.txt
reading manifest file 'Momoko.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'Momoko.egg-info/SOURCES.txt'
running build_ext
test_connection_factory (tests.MomokoFactoriesTest)
Testing that connection_factory parameter is properly propagated ... ok
test_cursor_factory (tests.MomokoFactoriesTest)
Testing that cursor_factory parameter is properly propagated ... ok
test_parallel_queries (tests.MomokoParallelTest)
Testing that pool queries database in parallel ... ok
test_parallel_queries_after_reconnect_all (tests.MomokoParallelTest)
Testing that pool still queries database in parallel after ALL connections were killed ... FAIL
test_parallel_queries_after_reconnect_some (tests.MomokoParallelTest)
Testing that pool still queries database in parallel after SOME connections were killed ... FAIL
test_server_side_cursor (tests.MomokoServerSideCursorTest)
Testing server side cursors support ... ok
test_setsession (tests.MomokoSetsessionTest)
Testing that setssion parameter is honoured ... ok
test_dont_stretch (tests.MomokoStretchTest)
Testing that we do not stretch unless needed ... ok
test_dont_stretch_after_reconnect (tests.MomokoStretchTest)
Testing that reconnecting dead connection does not trigger pool stretch ... ok
test_parallel_queries (tests.MomokoStretchTest)
Run parallel queies and check that pool size matches number of jobs ... ok
test_parallel_queries_after_reconnect_all (tests.MomokoStretchTest)
Testing that pool still queries database in parallel after ALL connections were killed ... FAIL
test_parallel_queries_after_reconnect_some (tests.MomokoStretchTest)
Testing that pool still queries database in parallel after SOME connections were killed ... FAIL
test_stretch_after_disonnect (tests.MomokoStretchTest)
Testing that stretch works after disconnect ... FAIL
test_stretch_genconn (tests.MomokoStretchTest)
Testing that stretch works together with get/putconn ... ok
test_callproc (tests.MomokoTest)
Testing callproc ... ok
test_getconn_manage (tests.MomokoTest)
Testing getconn + context manager functionality ... ok
test_getconn_putconn (tests.MomokoTest)
Testing getconn/putconn functionality ... ok
test_large_query (tests.MomokoTest)
Testing support for large queries ... ok
test_mogrify (tests.MomokoTest)
Testing mogrify ... ok
test_mogrify_error (tests.MomokoTest)
Testing that mogri propagates exception properly ... ok
test_op (tests.MomokoTest)
Testing Op ... ok
test_op_early_exception (tests.MomokoTest)
Testing that Op propagates early exceptions properly ... ok
test_op_exception (tests.MomokoTest)
Testing that Op propagates exception properly ... ok
test_query_error (tests.MomokoTest)
Testing that execute method propages exception properly ... ok
test_single_query (tests.MomokoTest)
Testing single query ... ok
test_transaction (tests.MomokoTest)
Testing transaction functionality ... ok
test_transaction_rollback (tests.MomokoTest)
Testing transaction auto-rollback functionality ... ok
test_transaction_with_reconnect (tests.MomokoTest)
Test whether transaction works after reconnect ... ok
test_unicode_transaction (tests.MomokoTest)
Testing transaction functionality ... ok
test_wait_all_ops (tests.MomokoTest)
Testing WaitAllOps ... ok
test_wait_all_ops_exception (tests.MomokoTest)
Testing that WaitAllOps propagates exception properly ... ok
test_wait_op (tests.MomokoTest)
Testing WaitOp ... ok
test_wait_op_exception (tests.MomokoTest)
Testing that WaitOp propagates exception properly ... ok
test_reconnect (tests.MomokoVolatileDbTest)
Testing if we can reconnect if connections die ... ok
test_reconnect_interval_bad_path (tests.MomokoVolatileDbTest)
Testing that pool does not try to reconnect right after last connection attempt failed ... ok
test_reconnect_interval_good_path (tests.MomokoVolatileDbTest)
Testing that we can recover if database was down during startup ... ok
test_startup (tests.MomokoVolatileDbTest)
Testing that all connections are dead after pool init with bad dsn ... ok
test_startup_local (tests.MomokoVolatileDbTest)
Testing that we catch early exeception with local connections ... ok

FAIL: test_parallel_queries_after_reconnect_all (tests.MomokoParallelTest)

Testing that pool still queries database in parallel after ALL connections were killed

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in call
result = self.orig_method(_args, _kwargs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 477, in test_parallel_queries_after_reconnect_all
self.test_parallel_queries()
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 472, in test_parallel_queries
self.assertLess(execution_time, sleep_time1.10, msg="Query execution was too long")
AssertionError: Query execution was too long

FAIL: test_parallel_queries_after_reconnect_some (tests.MomokoParallelTest)

Testing that pool still queries database in parallel after SOME connections were killed

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in call
result = self.orig_method(_args, _kwargs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 483, in test_parallel_queries_after_reconnect_some
self.test_parallel_queries()
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 472, in test_parallel_queries
self.assertLess(execution_time, sleep_time1.10, msg="Query execution was too long")
AssertionError: Query execution was too long

FAIL: test_parallel_queries_after_reconnect_all (tests.MomokoStretchTest)

Testing that pool still queries database in parallel after ALL connections were killed

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in call
result = self.orig_method(_args, _kwargs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 477, in test_parallel_queries_after_reconnect_all
self.test_parallel_queries()
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 493, in test_parallel_queries
super(MomokoStretchTest, self).test_parallel_queries(jobs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 472, in test_parallel_queries
self.assertLess(execution_time, sleep_time1.10, msg="Query execution was too long")
AssertionError: Query execution was too long

FAIL: test_parallel_queries_after_reconnect_some (tests.MomokoStretchTest)

Testing that pool still queries database in parallel after SOME connections were killed

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in call
result = self.orig_method(_args, _kwargs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 483, in test_parallel_queries_after_reconnect_some
self.test_parallel_queries()
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 493, in test_parallel_queries
super(MomokoStretchTest, self).test_parallel_queries(jobs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 472, in test_parallel_queries
self.assertLess(execution_time, sleep_time1.10, msg="Query execution was too long")
AssertionError: Query execution was too long

FAIL: test_stretch_after_disonnect (tests.MomokoStretchTest)

Testing that stretch works after disconnect

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in call
result = self.orig_method(_args, _kwargs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 509, in test_stretch_after_disonnect
self.test_parallel_queries()
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 493, in test_parallel_queries
super(MomokoStretchTest, self).test_parallel_queries(jobs)
File "/home/kamel/vagrant-test/new_labola/tornado-boilerplate/model/momoko/tests.py", line 472, in test_parallel_queries
self.assertLess(execution_time, sleep_time1.10, msg="Query execution was too long")
AssertionError: Query execution was too long

---

Ran 38 tests in 28.159s

FAILED (failures=5)

[haizaar]

Where does you postgres server run? Localhost?

If not on localhost, can you ping it and post the output?

[kamelzcs]

It was on the localhost.

db_database = os.environ.get('MOMOKO_TEST_DB', 'test')
db_user = os.environ.get('MOMOKO_TEST_USER', 'postgres')
db_password = os.environ.get('MOMOKO_TEST_PASSWORD', 'foo')
db_host = os.environ.get('MOMOKO_TEST_HOST', 'localhost')
db_port = os.environ.get('MOMOKO_TEST_PORT', 5432)

[haizaar]

I think somehow in your case connections take long to reconnect. Regular parallel test in your case goes fine, but reconnect test does not.

I can bump assert time to 1.3 factor, but I wonder if its a good idea.

May be you should measure how long on average it takes to initialize one synchronous connection (regular psycopg2.connect) in your setup. Probably it will shed some light.