Refactor certain doctests towards either reStructuredText documentation or Python tests

.coveragerc

@@ -7,4 +7,5 @@ omit =
  parts/*
  eggs/*
 exclude_lines =
+ # pragma: no cover
  raise NotImplemented

.gitignore

@@ -18,3 +18,4 @@ eggs/
 htmlcov/
 out/
 parts/
+tmp/

DEVELOP.rst

@@ -36,6 +36,25 @@ Ignore specific test directories::
 
  ./bin/test -vvvv --ignore_dir=testing
 
+The ``LayerTest`` test cases have quite some overhead. Omitting them will save
+a few cycles (~70 seconds runtime)::
+
+ ./bin/test -t '!LayerTest'
+
+Invoke all tests without integration tests (~15 seconds runtime)::
+
+ ./bin/test --layer '!crate.testing.layer.crate' --test '!LayerTest'
+
+Yet ~130 test cases, but only ~5 seconds runtime::
+
+ ./bin/test --layer '!crate.testing.layer.crate' --test '!LayerTest' \
+ -t '!test_client_threaded' -t '!test_no_retry_on_read_timeout' \
+ -t '!test_wait_for_http' -t '!test_table_clustered_by'
+
+To inspect the whole list of test cases, run::
+
+ ./bin/test --list-tests
+
 You can run the tests against multiple Python interpreters with `tox`_::
 
  tox

bin/test

@@ -11,6 +11,7 @@ base = os.path.dirname(base)
 sys.argv[0] = os.path.abspath(sys.argv[0])
 
 if __name__ == '__main__':
- zope.testrunner.run((['--auto-color', '--verbose']) + [
- '--test-path', join(base, 'src'),
- ])
+ zope.testrunner.run([
+ '-vvv', '--auto-color',
+ '--test-path', join(base, 'src')],
+ )

codecov.yml

@@ -3,7 +3,12 @@
 
 coverage:
  status:
+
  project:
  default:
- target: 85% # the required coverage value
- threshold: 3% # the leniency in hitting the target
+ target: auto # the required coverage value
+ threshold: 3% # the leniency in hitting the target
+
+ patch:
+ default:
+ informational: true

src/crate/client/doctests/blob.txt → docs/by-example/blob.rst

@@ -1,9 +1,13 @@
-==============
-Blob Container
-==============
+==================
+Blob container API
+==================
+
+The connection object provides a convenience API for easy access to `blob
+tables`_.
 
-The connection provides a blob container implementation for easy use of the
-bloba API.
+
+Get blob container handle
+=========================
 
 Create a connection::
 
@@ -15,14 +19,14 @@ Get a blob container::
  >>> container = client.get_blob_container('myfiles')
 
 
-Store Blobs
+Store blobs
 ===========
 
-The container allows to store a blob without explicitely providing the hash
+The container allows to store a blob without explicitly providing the hash
 for the blob. This feature is possible if the blob is provided as a seekable
 stream like object.
 
-Store a StringIO stream::
+Store a ``StringIO`` stream::
 
  >>> from io import BytesIO
  >>> f = BytesIO(b'StringIO data')
@@ -60,7 +64,7 @@ Check for existence
  False
 
 
-Retrieve Blobs
+Retrieve blobs
 ==============
 
 Blobs can be retrieved using its hash::
@@ -73,7 +77,7 @@ Blobs can be retrieved using its hash::
  True
 
 
-Delete Blobs
+Delete blobs
 ============
 
 Blobs can be deleted using its hash::
@@ -88,10 +92,12 @@ Trying to delete a not existing blob::
  >>> container.delete(string_blob)
  False
 
-Close Connection
+Close connection
 ================
 
 Close the connection to clear the connection pool::
 
  >>> client.close()
 
+
+.. _blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html

src/crate/client/doctests/client.txt → docs/by-example/client.rst

@@ -1,8 +1,19 @@
-============
-Client Usage
-============
+===============
+Database client
+===============
 
-Connect to a Database
+``crate.client.connect`` is the primary method to connect to CrateDB using
+Python. This section of the documentation outlines different methods to connect
+to the database cluster, as well as how to run basic inquiries to the database,
+and closing the connection again.
+
+.. rubric:: Table of Contents
+
+.. contents::
+ :local:
+
+
+Connect to a database
 =====================
 
 Before we can start we have to import the client::

src/crate/client/doctests/connection.txt → docs/by-example/connection.rst

@@ -1,17 +1,35 @@
-==========
-Connection
-==========
+=====================
+The Connection object
+=====================
 
-::
+This documentation section outlines different attributes, methods, and
+behaviors of the ``crate.client.connection.Connection`` object.
+
+The examples use an instance of ``ClientMocked`` instead of a real ``Client``
+instance. This allows us to verify the examples without needing a real database
+connection.
+
+.. rubric:: Table of Contents
+
+.. contents::
+ :local:
 
- >>> from crate.client import connect
 
-We create a new connection object::
+connect()
+=========
+
+This section sets up a connection object, and inspects some of its attributes.
+
+ >>> from crate.client import connect
+ >>> from crate.client.test_util import ClientMocked
 
- >>> connection = connect(client=connection_client_mocked)
+ >>> connection = connect(client=ClientMocked())
  >>> connection.lowest_server_version.version
  (2, 0, 0)
 
+cursor()
+========
+
 Calling the ``cursor()`` function on the connection will
 return a cursor object::
 
@@ -23,6 +41,9 @@ cursor object::
  >>> cursor.rowcount
  -1
 
+close()
+=======
+
 Now we close the connection::
 
  >>> connection.close()
@@ -45,9 +66,3 @@ raise a ``ProgrammingError``::
  Traceback (most recent call last):
  ...
  crate.client.exceptions.ProgrammingError: Connection closed
-
- >>> connection = connect(client=connection_client_mocked)
- >>> cursor = connection.cursor()
- >>> cursor.rowcount
- -1
-

src/crate/client/doctests/cursor.txt → docs/by-example/cursor.rst

@@ -1,35 +1,44 @@
-======
-Cursor
-======
+=================
+The Cursor object
+=================
 
+This documentation section outlines different attributes, methods, and
+behaviors of the ``crate.client.cursor.Cursor`` object.
 
-Setup
-=====
+The example code uses ``ClientMocked`` and ``set_next_response`` for
+demonstration purposes, so they don't need a real database connection.
+
+.. rubric:: Table of Contents
+
+.. contents::
+ :local:
+
+
+Introduction
+============
 
 This section sets up a cursor object, inspects some of its attributes, and sets
 up the response for subsequent cursor operations.
 
-::
-
  >>> from crate.client import connect
  >>> from crate.client.converter import DefaultTypeConverter
  >>> from crate.client.cursor import Cursor
+ >>> from crate.client.test_util import ClientMocked
 
- >>> connection = connect(client=connection_client_mocked)
+ >>> connection = connect(client=ClientMocked())
  >>> cursor = connection.cursor()
 
-The rowcount and duration attribute is -1 in case no ``execute()`` has been performed on the cursor.
-
-::
+The ``rowcount`` and ``duration`` attributes are ``-1``, in case no ``execute()`` has
+been performed on the cursor yet.
 
  >>> cursor.rowcount
  -1
 
  >>> cursor.duration
  -1
 
-Hardcode the next response of the mocked connection client, so we won't need a sql statement
-to execute::
+Define the response of the mocked connection client. It will be returned on
+request without needing to execute an SQL statement.
 
  >>> connection.client.set_next_response({
  ... "rows":[ [ "North West Ripple", 1 ], [ "Arkintoofle Minor", 3 ], [ "Alpha Centauri", 3 ] ],
@@ -168,8 +177,6 @@ Iterating over a new cursor without results will immediately raise a Programming
 description
 ===========
 
-::
-
  >>> cursor.description
  (('name', None, None, None, None, None, None), ('position', None, None, None, None, None, None))
 
@@ -238,8 +245,8 @@ The attribute is -1 in case the cursor has been closed::
  ... "cols":[ "name", "position" ],
  ... })
 
-executemany
-===========
+executemany()
+=============
 
 ``executemany()`` allows to execute a single sql statement against a sequence
 of parameters::
@@ -340,15 +347,15 @@ inspect the ``DataType`` enum, or the documentation about the list of available
 `CrateDB data type identifiers for the HTTP interface`_.
 
 To create a simple converter for converging CrateDB's ``BIT`` type to Python's
-``int`` type::
+``int`` type.
 
  >>> from crate.client.converter import Converter, DataType
 
  >>> converter = Converter({DataType.BIT: lambda value: int(value[2:-1], 2)})
  >>> cursor = connection.cursor(converter=converter)
 
 Proof that the converter works correctly, ``B\'0110\'`` should be converted to
-``6``. CrateDB's ``BIT`` data type has the numeric identifier ``25``::
+``6``. CrateDB's ``BIT`` data type has the numeric identifier ``25``.
 
  >>> connection.client.set_next_response({
  ... "col_types": [25],
@@ -374,8 +381,6 @@ desired time zone.
 For your reference, in the following examples, epoch 1658167836758 is
 ``Mon, 18 Jul 2022 18:10:36 GMT``.
 
-::
-
  >>> import datetime
  >>> tz_mst = datetime.timezone(datetime.timedelta(hours=7), name="MST")
  >>> cursor = connection.cursor(time_zone=tz_mst)
@@ -402,7 +407,7 @@ The available options are:
 - ``zoneinfo.ZoneInfo("Australia/Sydney")``
 - ``+0530`` (UTC offset in string format)
 
-Let's exercise all of them::
+Let's exercise all of them:
 
  >>> cursor.time_zone = datetime.timezone.utc
  >>> cursor.execute('')