Merge pull request #1216 from open-contracting/866-clarify-contractin…

…g-process-record-define-planning-process

Clarify contracting process, record and ocid, define planning process (866)

docs/getting_started/quality.md

@@ -29,7 +29,7 @@ Understanding all of the challenges above, we understand that increasing the tra
 
 All OCDS publications ought to meet the following criteria:
 
-1. **Registered**: The data uses a [registered OCID prefix](../schema/identifiers.md#contracting-process-identifier-ocid).
+1. **Registered**: The data uses a [registered OCID prefix](../schema/identifiers.md#open-contracting-process-identifier-ocid).
 1. **Discoverable**: It is possible to discover the data by navigating a website whose homepage is indexed by popular web search engines.
 1. **Retrievable**: It is possible to automate the download of all the data, either using an HTML page listing bulk download URLs, or using only machine-readable data as input.
 1. **Reviewable**: The [OCDS Data Review Tool](https://standard.open-contracting.org/review/) is able to report results on the data.

docs/getting_started/releases_and_records.md

@@ -47,7 +47,7 @@ Releases follow the [release schema](../schema/reference). The schema covers the
 
 #### Identifiers
 
-Each release contains an [ocid](../schema/identifiers.md#contracting-process-identifier-ocid) to identify the contracting process it relates to. An ocid is composed of a prefix and an unique process identifier chosen by the publisher.
+Each release contains an [ocid](../schema/identifiers.md#open-contracting-process-identifier-ocid) to identify the contracting process it relates to. An ocid is composed of a prefix and an unique process identifier chosen by the publisher.
 
 A release also needs a release identifier, unique in the scope of the contracting process. A release id can be built in several ways. Publishers can use any generation strategy, as long as the ids don’t collide within the same process.
 

docs/guidance/build.md

@@ -16,7 +16,7 @@ As you complete this phase, you can:
 
 ## Register an OCID prefix
 
-The [identifiers](../../schema/identifiers) reference page describes the contracting process identifier (`ocid`) and how ocid prefixes are used to ensure `ocid`s are globally unique.
+The [identifiers](../../schema/identifiers) reference page describes the open contracting process identifier (`ocid`) and how ocid prefixes are used to ensure `ocid`s are globally unique.
 
 To publish OCDS data, you need to register an ocid prefix.
 

docs/guidance/build/change_history.md

@@ -128,7 +128,7 @@ So far, the council used a single procurement system to manage the process. The
 
 The council now uses a separate financial system to manage payments. The financial system publishes the new OCDS release.
 
-The procurement system and the financial system share a common contracting process identifier. This means that the two systems can publish releases using the same `ocid`.
+The procurement system and the financial system share a common identifier for contracting processes. This means that the two systems can publish releases using the same `ocid`.
 
 The new release uses the 'implementation' tag. The `contracts.implementation.transactions` section includes the details of the payment.
 

docs/guidance/build/hosting.md

@@ -34,9 +34,9 @@ When the suggested limits entail publication of multiple files, publishers ought
 For releases, publishers can:
 
 1. Segment by **release date** - placing all the releases from a given day, month or year in the same file;
-1. Segment by **contracting process identifier** - placing all the releases related to a given set of contract process identifiers together in the same package;
+1. Segment by **open contracting process identifier** - placing all the releases related to a given set of process identifiers together in the same package;
 
-For records, publishers can segment by the first **release date** associated with a contracting process, or by **contracting process identifier.**
+For records, publishers can segment by the first **release date** associated with a contracting process, or by **open contracting process identifier.**
 
 Following these approaches will avoid release and records 'jumping' between files when the bulk files are updated. 
 

docs/guidance/map.md

@@ -80,6 +80,7 @@ Mapping data to OCDS is not always obvious. Please refer to our how-to guides an
 :maxdepth: 2
 :titlesonly:
 
+map/contracting_planning_processes
 map/unsuccessful_processes
 map/related_processes
 map/pre-qualification

docs/guidance/map/contracting_planning_processes.md

@@ -0,0 +1,29 @@
+# Contracting processes and planning processes
+
+OCDS recognizes two types of processes: contracting processes and planning processes. In OCDS, a given process is uniquely identified by an [open contracting process identifier](../../schema/identifiers.md#open-contracting-process-identifier-ocid) (`ocid`). This section helps map your contracting activities (most often procurement procedures) to their OCDS representation.
+
+OCDS defines a contracting process as:
+
+> All the actions aimed at implementing one or more contracts. This covers tendering, awarding, contracting and implementation. It does not include actions linked to planning, as these are often less structured and may be linked to multiple contracting processes. In multiple stage procedures (e.g. framework agreements with reopening of competition), each round of competition is treated as a separate contracting process.
+
+> Procedures that failed and were restarted are considered new processes.
+
+> Boundaries between processes (e.g. whether two contracts result from a single process or from two processes) are set by buyers depending on their needs (e.g. efficient division of labor, clear communication with the market) and legislation (e.g. rules on using procedures and lots).
+
+OCDS defines a planning process as: 
+
+> All the actions aimed at planning one or more contracting processes. This covers, for example, establishing the rationale for the procurement, giving the market a general description of the purchase, getting the necessary budget, forecasting and conducting market research.
+
+> Planning processes are often less structured than contracting processes, so one or more planning processes may lead to one or more contracting processes.
+
+![Contracting Process](../../_static/png/contracting_process.png)
+
+A planning process ought to have its `releaseTag` set to 'planning' (or 'planningUpdate'). A contracting process can have `releaseTag` set to [any other value from the codelist](../../schema/codelists.md#release-tag). A planning process should not contain the `releaseTag` 'tender' even if it contains a `tender` object. The two processes ought to be linked together using the `relatedProcesses` array in the releases for the contracting process, with the 'planning' code in the related process' `relationship` array.
+
+```{note}
+We recommend publishing data about planning and contracting processes under separate `ocid`s, following the definitions above. That said, publications that combine planning and contracting data under a single `ocid` remain conformant in OCDS 1.2. A required separation can be considered for OCDS 2.0.
+```
+
+```{note}
+In OCDS 1.2 and earlier, it is not possible to publish all information about multi-stage procedures under a single `ocid`. There is guidance on how to deal with this for [framework agreements](related_processes) and for [pre-qualification and pre-selection](pre-qualification). If you want to disclose this type of information (including other types of multi-stage procedures, such as competitive dialogues and innovation partnerships), [contact the OCDS Helpdesk](../../support/index). The approach to modelling multi-stage procedures in a future, backwards-incompatible version of the standard is under discussion on [GitHub](https://github.com/open-contracting/standard/issues/440).
+```

docs/guidance/map/pre-qualification.md

@@ -1,4 +1,4 @@
-# Pre-qualification and pre-selection
+# Processes with pre-qualification and pre-selection
 
 In single-stage procedures, procuring entities invite suppliers to bid without submitting any prior information.  Such procedures are straightforward to model in OCDS.
 

docs/guidance/map/related_processes.md

@@ -54,10 +54,7 @@ In OCDS, a contracting process brings together, under a single identifier, the i
 * What was the total value of spending that resulted from this award?
 * Was a renewal of this contract signed?
 
-In some cases, complex contracting processes cannot be represented under a single identifier under OCDS' model, because:
-
-* There are multiple competitive stages: for example, when a framework agreement involves second-stage competitions.
-* The procurement systems used at different stages of the process are managed by different bodies, and cannot be integrated.
+In some cases, complex contracting processes cannot be represented under a single identifier, because there are multiple stages. For example, this is the case when a framework is set up, and then mini-competitions are used for purchases from the framework.
 
 OCDS models the first and second stages of framework agreement procedures as separate contracting processes, linked together using the `relatedProcesses` array. The `tender.techniques.hasFrameworkAgreement` field, from the [Techniques](https://extensions.open-contracting.org/en/extensions/techniques/master/) extension, is used to identify contracting processes that represent the first stage of a framework agreement procedure. The presence of a related process with a `.relationship` set to 'framework' is used to identify contracting processes that represent the second stage of a framework agreement procedure.
 

docs/guidance/map/unsuccessful_processes.md

@@ -1,22 +1,8 @@
 # Unsuccessful processes
 
-In the case of procurement, a contracting process can be defined as a procurement procedure. There is a one-to-one correspondence between the first stage of a procurement procedure (tender) and a contracting process.
-
-In OCDS, at a conceptual level, a contracting process is intended to match each concrete attempt to start a procedure that leads to one or more contracts. Attempts can include: an invitation to tender (in open procedure); an invitation to request to participate; a competition for a concession; a direct award, etc.
-
-![Contracting Process](../../_static/png/contracting_process.png)
-
-In OCDS, the `ocid` is the unique identifier for a contracting process. As the initiation of the procurement process is the tender, normally the identifier for a tender can be used as the `ocid`.
-
 In most jurisdictions, if a procedure is cancelled or unsuccessful, and a **new procedure** is started to procure the same items, the two procedures are considered two **different** contracting processes. This is in keeping with the OCDS definition of a contracting process.
 
-But in other jurisdictions, such as Paraguay, the planning stage is considered as the initiation of the process. In these jurisdictions when a tender fails and a new tender is started, the two tenders are considered part of the same contracting process. This differs from the OCDS definition of a contracting process.
-
-In OCDS, it is relevant and desirable to include the planning information that relates to the process, but the contracting process is not interpreted as ‘starting’ with the planning stage. In OCDS, the planning stage is something that comes **before** the initiation of a contracting process. The initiation of the procedure is not the planning stage, because at least one of the following is true of a planning stage: it is not a concrete attempt to award one or more contracts like a request for tender, etc.; it is not a concrete opportunity for potential suppliers to participate in; it does not describe the competitive conditions.
-
-However a jurisdiction treats unsuccessful tenders and subsequent tenders, in OCDS they are considered separate but related contracting processes.
-
-This relationship can be modelled using the `relatedProcess` array at the release level, with the ‘unsuccessfulProcess’ relationship type.  
+However, in some jurisdictions, such as Paraguay, the planning stage is considered as the initiation of the process. In these jurisdictions, when a tender fails and a new tender is started, the two tenders are considered part of the same contracting process. This differs from the OCDS definition of a contracting process. OCDS, instead, records the cancelled procedure in the `relatedProcesses` array of the new procedure, with the 'unsuccessfulProcess' code in the related process' `relationship` array.  
 
 ![Unsuccessful Tender](../../_static/png/unsuccessful-tender.png)
 
@@ -48,7 +34,7 @@ To construct an `ocid` for the second contracting process, Paraguay adds a conse
 
 Paraguay could also have used the identifier for the second tender as the `ocid` for the second contracting process.
 
-The `relatedProcess` block links the two processes, with the relationship set to ‘unsuccessfulProcess’.
+The `relatedProcesses` block links the two processes, with the relationship set to ‘unsuccessfulProcess’.
 
 ```{jsoninclude} ../../examples/unsuccessful-tender-related-process.json
 :jsonpointer:

docs/history/changelog.md

@@ -14,7 +14,7 @@ Per the [normative and non-normative content and changes policy](https://docs.go
 
 * Guidance section:
   * [#986](https://github.com/open-contracting/standard/pull/986) Add implementation guidance from OCP website.
-  * Add worked examples for the Map phase [#947](https://github.com/open-contracting/standard/pull/947) [#948](https://github.com/open-contracting/standard/pull/948) [#950](https://github.com/open-contracting/standard/pull/950) [#974](https://github.com/open-contracting/standard/pull/974) [#990](https://github.com/open-contracting/standard/pull/990) [#999](https://github.com/open-contracting/standard/pull/999) [#1007](https://github.com/open-contracting/standard/pull/1007) [#1123](https://github.com/open-contracting/standard/pull/1123).
+  * Add worked examples for the Map phase [#947](https://github.com/open-contracting/standard/pull/947) [#948](https://github.com/open-contracting/standard/pull/948) [#950](https://github.com/open-contracting/standard/pull/950) [#974](https://github.com/open-contracting/standard/pull/974) [#990](https://github.com/open-contracting/standard/pull/990) [#999](https://github.com/open-contracting/standard/pull/999) [#1007](https://github.com/open-contracting/standard/pull/1007) [#1123](https://github.com/open-contracting/standard/pull/1123) [#1216](https://github.com/open-contracting/standard/pull/1216).
   * Add worked examples for the Build phase [#951](https://github.com/open-contracting/standard/pull/951) [#997](https://github.com/open-contracting/standard/pull/997).
   * [#963](https://github.com/open-contracting/standard/pull/963) Remove guidance on web discovery.
   * [#986](https://github.com/open-contracting/standard/pull/986) Merge Registration page into Build page.
@@ -101,6 +101,13 @@ Per the [normative and non-normative content and changes policy](https://docs.go
 
 ### Schema
 
+* Clarify core concepts:
+  * [#1216](https://github.com/open-contracting/standard/pull/1216) Define contracting process and planning process in the schema description. Update definition of release, record and ocid. Update references to contracting process so that it takes take the planning process into account.
+  * [#1182](https://github.com/open-contracting/standard/pull/1182) `buyer`
+  * [#1163](https://github.com/open-contracting/standard/pull/1163) `tender.procuringEntity`
+  * [#1232](https://github.com/open-contracting/standard/pull/1232) `awards.suppliers`
+  * [#1208](https://github.com/open-contracting/standard/pull/1208) `contracts` and its fields
+
 * Add new fields:
   * [#1125](https://github.com/open-contracting/standard/pull/1125) `Item.unit.weight`
   * [#1165](https://github.com/open-contracting/standard/pull/1165) `statusDetails` to `Tender`, `Award` and `Contract`
@@ -117,17 +124,6 @@ Per the [normative and non-normative content and changes policy](https://docs.go
   * [#1200](https://github.com/open-contracting/standard/pull/1200) `tender.submissionMethod`, because all codes from the `submissionMethod` codelist are deprecated.
   * [#1296](https://github.com/open-contracting/standard/pull/1296) `tender.eligibilityCriteria` in favor of the new `tender.exclusionGrounds` field, in order to use more common terminology and improve semantics. 
 
-* Clarify core concepts:
-  * [#1182](https://github.com/open-contracting/standard/pull/1182) `buyer`
-  * [#1163](https://github.com/open-contracting/standard/pull/1163) `tender.procuringEntity`
-  * [#1232](https://github.com/open-contracting/standard/pull/1232) `awards.suppliers`
-  * [#1208](https://github.com/open-contracting/standard/pull/1208) `contracts` and its fields
-
-* Clarify merging behavior:
-  * [#1242](https://github.com/open-contracting/standard/pull/1242) Clarify that the releases to merge must use the same version of OCDS.
-  * [#1242](https://github.com/open-contracting/standard/pull/1242) Narrow the uniqueness scope of a release's `id` to its `ocid` and OCDS version (was `ocid` only), to allow the publication of the same release for different versions of OCDS.
-  * [#1315](https://github.com/open-contracting/standard/pull/1315) Update the descriptions of `id` and `date`, to add rules for compiled releases.
-
 * Update and clarify field descriptions:
   * [#1094](https://github.com/open-contracting/standard/pull/1094) `Organization.id`, to clarify its uniqueness.
   * [#1113](https://github.com/open-contracting/standard/pull/1113) `ocid`, to recommend a hyphen after the ocid prefix.
@@ -145,6 +141,11 @@ Per the [normative and non-normative content and changes policy](https://docs.go
   * [#1112](https://github.com/open-contracting/standard/pull/1112) `Period.durationInDays`:  "If a startDate and endDate are set, this field, if used, **must** be equal to the difference between startDate and endDate. Otherwise, if a startDate and maxExtentDate are set, this field, if used, **must** be equal to the difference between startDate and maxExtentDate." ("should" replaced with "must")
   * [#1112](https://github.com/open-contracting/standard/pull/1112) `Contract.items`: "If the items contracted are identical to the items awarded, this field **should** be omitted." (rephrased)
 
+* Clarify merging behavior:
+  * [#1242](https://github.com/open-contracting/standard/pull/1242) Clarify that the releases to merge must use the same version of OCDS.
+  * [#1242](https://github.com/open-contracting/standard/pull/1242) Narrow the uniqueness scope of a release's `id` to its `ocid` and OCDS version (was `ocid` only), to allow the publication of the same release for different versions of OCDS.
+  * [#1315](https://github.com/open-contracting/standard/pull/1315) Update the descriptions of `id` and `date`, to add rules for compiled releases.
+
 * Make minor changes to the schema's organization:
   * [#1240](https://github.com/open-contracting/standard/pull/1240) Move `Unit` from `Item.unit` to the schema definitions.
   * [#1354](https://github.com/open-contracting/standard/pull/1354) Switch the positions of `contract.dateSigned` and `contract.period` to correspond with the order in `Award`.
@@ -169,6 +170,7 @@ Per the [normative and non-normative content and changes policy](https://docs.go
 * [#1161](https://github.com/open-contracting/standard/pull/1161) Change recommendation for unknown time component.
 * [#1189](https://github.com/open-contracting/standard/pull/1189) Add recommendations about publishing and referencing documents in the document reference section.
 * [#1208](https://github.com/open-contracting/standard/pull/1208) Update guidance with new field definitions.
+* [#1216](https://github.com/open-contracting/standard/pull/1216) Update definitions of contracting process, record, and ocid. Introduce definition of planning process.
 * [#1307](https://github.com/open-contracting/standard/pull/1307) Clarify uniqueness rules for records.
 * [#1315](https://github.com/open-contracting/standard/pull/1315) Add rules on setting `id` and `date` for compiled releases to the merging specification.