diff --git a/.gitignore b/.gitignore index 479fd54c4654..1ff962a9183b 100644 --- a/.gitignore +++ b/.gitignore @@ -68,7 +68,7 @@ coverage.xml # Sphinx documentation docs/_build/ docs/.doctrees -docsource/modules*-*.rst +docsource/coverage_analysis/modules*-*.rst # Backup files *~ diff --git a/build_openupgrade_docs b/build_openupgrade_docs index 2067305a6a55..2dda8413042c 100755 --- a/build_openupgrade_docs +++ b/build_openupgrade_docs @@ -2,6 +2,7 @@ OUTPUT_DIR=$(realpath $(dirname $0)/./docs) SOURCE_DIR=$(realpath $(dirname $0)/./docsource) +COVERAGE_DIR=$(realpath $(dirname $0)/./docsource/coverage_analysis) BASE_URL=https://raw.githubusercontent.com/OCA/OpenUpgrade/ DOC_PARTS="7.0/openerp/openupgrade/doc/source/modules50-60.rst @@ -23,7 +24,7 @@ for line in $DOC_PARTS; do continue fi name=$(basename $line) - wget -q $BASE_URL/$line -O $SOURCE_DIR/$name + wget -q $BASE_URL/$line -O $COVERAGE_DIR/$name done mkdir -p $OUTPUT_DIR/source/.static diff --git a/docsource/intro.rst b/docsource/010_introduction.rst similarity index 100% rename from docsource/intro.rst rename to docsource/010_introduction.rst diff --git a/docsource/020_required_knowledge.rst b/docsource/020_required_knowledge.rst new file mode 100644 index 000000000000..270b68734c3d --- /dev/null +++ b/docsource/020_required_knowledge.rst @@ -0,0 +1,4 @@ +Required Knowledge +================== + +TODO : write here prerequisites. \ No newline at end of file diff --git a/docsource/030_coverage_analysis.rst b/docsource/030_coverage_analysis.rst new file mode 100644 index 000000000000..ff43e1b36e3d --- /dev/null +++ b/docsource/030_coverage_analysis.rst @@ -0,0 +1,29 @@ +Coverage Analysis +================= + +For all official addons, migration scripts should be provided for migrating +between subsequent major releases of Odoo. This is the goal of the project. +The design is such that the effort can be distributed between different +developing parties. See below for an overview of module coverage per version. + +Check if there are migration scripts provided for the set of modules that +are installed in your Odoo database. If there are modules for which no +migration scripts have been developed yet, your migration may fail or the +integrity of your database may be lacking. +Refer to the :doc:`080_migration_script_development` documentation to add the missing migration scripts. + +.. toctree:: + :maxdepth: 1 + + coverage_analysis/modules150-160 + coverage_analysis/modules140-150 + coverage_analysis/modules130-140 + coverage_analysis/modules120-130 + coverage_analysis/modules110-120 + coverage_analysis/modules100-110 + coverage_analysis/modules90-100 + coverage_analysis/modules80-90 + coverage_analysis/modules70-80 + coverage_analysis/modules61-70 + coverage_analysis/modules60-61 + coverage_analysis/modules50-60 diff --git a/docsource/migration_details.rst b/docsource/040_run_migration.rst similarity index 83% rename from docsource/migration_details.rst rename to docsource/040_run_migration.rst index 6fd46c850df0..c8e39b6b2262 100644 --- a/docsource/migration_details.rst +++ b/docsource/040_run_migration.rst @@ -33,23 +33,13 @@ openupgradelib pip install git+https://github.com/OCA/openupgradelib.git@master#egg=openupgradelib -2. Check coverage of the migration scripts for your installed modules -********************************************************************* - -Check if there are migration scripts provided for the set of modules that -are installed in your Odoo database. If there are modules for which no -migration scripts have been developed yet, your migration may fail or the -integrity of your database may be lacking. Check the module coverage in -this documentation under :doc:`status` and refer to the :doc:`development` -documentation to add the missing migration scripts. - -3. Make a copy of the database to migrate +2. Make a copy of the database to migrate ***************************************** Decide which database you are going to upgrade. You absolutely *must* make a backup of your live database before you start this process! -4. Adjust the configuration for Odoo and OpenUpgrade +3. Adjust the configuration for Odoo and OpenUpgrade **************************************************** Edit the configuration files and command line parameters to point to the @@ -90,7 +80,7 @@ file under a header *[openupgrade]* selected for installation (as will their dependencies). Be careful not to introduce a circular dependency using this directive. -5. Run the upgrade, fix data and repeat... +4. Run the upgrade, fix data and repeat... ****************************************** Run the upgrade and check for errors. You will probably learn a lot about @@ -104,4 +94,4 @@ Write the missing migration scripts At this stage, if some of your modules don't have yet migration scripts, you might need to add them yourself. -Read more about the development of migrations scripts in :doc:`development` +Read more about the development of migrations scripts in :doc:`080_migration_script_development` diff --git a/docsource/after_migration.rst b/docsource/050_after_migration.rst similarity index 100% rename from docsource/after_migration.rst rename to docsource/050_after_migration.rst diff --git a/docsource/migrationmanager.rst b/docsource/060_odoo_migration_manager.rst similarity index 59% rename from docsource/migrationmanager.rst rename to docsource/060_odoo_migration_manager.rst index 958b544dccaa..0dc7bb349118 100644 --- a/docsource/migrationmanager.rst +++ b/docsource/060_odoo_migration_manager.rst @@ -1,5 +1,5 @@ The Odoo Migration Manager -++++++++++++++++++++++++++ +========================== The core mechanism that OpenUpgrade relies on is the migration manager that is already built into Odoo itself. It is this mechanism that was used by Odoo to @@ -26,3 +26,25 @@ Note that you can use this mechanism for your own custom module's lifecycle outside of the OpenUpgrade context. For that reason, the OpenUpgrade helper methods were collected into the python openupgradelib that you can make available in any Odoo instance using the *pip* tool. + +FAQ +--- + +How are new dependencies treated by the Odoo migration manager? + New dependencies (like the *edi* module is a new dependency of the + *account* module) will be detected by the upgrade process. The + Odoo server code is slightly modified to loop over this part + of the process to install new dependencies and then return to + upgrading the modules that depend on them, until no more modules + are processed. + +Are migration scripts fired when installing new modules? + Yes. That includes any new dependencies that the new version of any + module might declare. You might want to check for a non true value + of the *version* argument, or (better) make your script robust to + running against a database that it does not apply to, in anticipation + of any unknown unknowns. Also another argument for not running the + OpenUpgrade server in production, even though we both know that you + would never ever do so anyway. Developers are free to corrupt the regular + workings of Odoo if it helps the migration. For instance, from OpenUpgrade + 9.0 on, the workflow engine is disabled during field recomputation. diff --git a/docsource/format.rst b/docsource/070_migration_files.rst similarity index 50% rename from docsource/format.rst rename to docsource/070_migration_files.rst index 105baf1a3f7f..cac5d1f75f7f 100644 --- a/docsource/format.rst +++ b/docsource/070_migration_files.rst @@ -1,11 +1,85 @@ -Fields analysis format -====================== +Migration Files +=============== -The first section displays models which are removed from the database. -The second section displays models which are added to the database. +For each odoo module, a **migration directory** contains the analysis of +the differences between the previous version and the current version +and the migration scripts developed to ensure the migration +of this module runs properly. -TODO: display which models moved to another module, instead of in the field -analysis? It should also be clear how to install such modules from the upgrade +For versions 14.0 and higher +---------------------------- + +Migration directories are hosted in `scripts` subdirectory of the +openupgrade_scripts module directory in the https://github.com/OCA/OpenUpgrade +repository. + +For example, the migration folder for migrating the `account` module +from version 13.0 to version 14.0 is located here : + +https://github.com/OCA/OpenUpgrade/tree/14.0/openupgrade_scripts/scripts/account/ + +For versions 13.0 and below +--------------------------- + +Migration directories are hosted in the `migration` folder of each module +in the forks of Odoo at https://github.com/OCA/OpenUpgrade. + + +For example, the migration folder for migrating the `account` module +from version 12.0 to version 13.0 is located here : + +https://github.com/OCA/OpenUpgrade/tree/13.0/addons/account/migrations + +Non official addons +------------------- + +For OCA or custom addons, migration directories are hosted in the `migration` +folder of each module. + +For example, the migration folder for migrating the OCA `account_global_discount` module +from version 12.0 to version 13.0 is located here : + +https://github.com/OCA/account-invoicing/tree/13.0/account_global_discount + +Directory contents +------------------ + +The contents of the migration directory per module are: + +* A file `openupgrade_analysis.txt` that contains data model changes and + changes in the set of XML records +* A file `openupgrade_analysis_work.txt` which is a copy of the previous file, + annotated by the developers of the migration scripts +* A file `noupdate_changes.xml` containing changes that may need to be loaded + during module migration (after reviewing the contents of this automatically + generated file). +* A file `pre-migration.py` (or any Python file(s) starting with `pre-`) that + is executed by the Odoo migration manager just before the module is loaded. +* A file `post-migration.py` (or any Python file(s) starting with `post-`) that + is executed by the Odoo migration manager right after the module was loaded. +* A file `end-migration.py` (or any Python file(s) starting with `end-`) that + is executed by the Odoo migration manager when all modules have been + processed. + +The migration directory of the base module contains an additional file +`openupgrade_general_log.txt`. This file contains some statistics as well +as the analysis records of modules that could not be found in the target +release of Odoo anymore. + +Generate the difference analysis files +-------------------------------------- + +You can create a custom set of analysis files on any set of modules between +any two versions of Odoo using the `upgrade_analysis` module that is hosted +in https://github.com/oca/server-tools. +(for versions 13.0 or lower, the module is named `openupgrade_records` and is +place in each branch of the OpenUpgrade repository under +`odoo/addons/openupgrade_records` path) + +Analysis files description +-------------------------- + +The first section displays models which are added or removed from the database. The second section lists the model fields which have been signalled by the analysis script. Every line lists the following columns: @@ -86,3 +160,27 @@ The change description flags the following types of change: The final section of the database layout analysis contains a simple report on the changes that were detected. + +XML IDs +------- + +The OpenUpgrade analysis files give a representation of the XML IDs that a +module defines, in comparison with the previous release of the module. + +XML IDs which do not occur in the updated version of all installed modules +will be removed automatically by the Odoo server, if they do not have +the noupdate attribute. Therefore, you can ignore most entries here, such as + + * ir.actions.* + * ir.model.fields + * ir.model.access + * ir.model + * ir.ui.* + * res.country* + * res.currency* + +When XML ids of such record types change, the record will be recreated under +the new id, and the old record will be unlinked. + +To manage changes to data defined with the noupdate flag, +please refer to the following use case :doc:`use_cases/xml_id_renaming`. \ No newline at end of file diff --git a/docsource/development.rst b/docsource/080_migration_script_development.rst similarity index 93% rename from docsource/development.rst rename to docsource/080_migration_script_development.rst index 768479e5cfb2..f6a142eb16ad 100644 --- a/docsource/development.rst +++ b/docsource/080_migration_script_development.rst @@ -4,10 +4,8 @@ Migration script development Overview -------- -The execution of the migration scripts is triggered by the "Migration Manager": -:doc:`migrationmanager`. - -As explained with more detail in the page :doc:`migrationmanager`, the migration scripts +As explained with more detail in the page "Migration Manager" +(:doc:`060_odoo_migration_manager`), the migration scripts of a module are usually the combination of 3 python files (might be less if some steps are not necessary): @@ -51,7 +49,7 @@ Learn from typical Use cases ---------------------------- .. toctree:: - :maxdepth: 2 + :maxdepth: 1 use_cases/field_fast_precomputation use_cases/field_renaming @@ -76,7 +74,7 @@ Since version 14, the migration scripts are located in: During the review for a given module, you can follow this process: -- Review the analysis (read more in :doc:`analysis`). First the file +- Review the analysis (read more in :doc:`070_migration_files`). First the file `openupgrade_analysis.txt` showing some differences between the 2 versions of the module. Then, `openupgrade_analysis_work.txt` to see what the developer planned to do. @@ -97,7 +95,7 @@ The Trial and error process for the development of your scripts --------------------------------------------------------------- Basically, this is the happening during the step when you try to run the upgrade -described in :doc:`migration_details`: +described in :doc:`040_run_migration`: - [A] get the copy of your database in old version as `db-upgrade`, it is easy to do through the database manager of your old odoo @@ -135,12 +133,6 @@ https://github.com/OCA/OpenUpgrade/pulls?q=is%3Apr+%5BFIX%5D+reset+exception Learning resources ------------------ -.. toctree:: - :maxdepth: 2 - - migrationmanager - devfaq - You can also refer to the following: - `OpenUpgrade API `_ diff --git a/docsource/community.rst b/docsource/090_contribute.rst similarity index 97% rename from docsource/community.rst rename to docsource/090_contribute.rst index 862e92415152..e0037433e7d3 100644 --- a/docsource/community.rst +++ b/docsource/090_contribute.rst @@ -1,5 +1,17 @@ +Contribute +========== + +In order to contribute to the OpenUpgrade project, please + +* Post your code contributions as pull requests on + https://github.com/OCA/OpenUpgrade +* Donate to the Odoo Community Association (https://github.com/sponsors/OCA) +* Hire any active contributor to this project to help you migrate your + database, and give back any code improvements developed during the project. + + Community involvement -===================== +--------------------- Writing migration scripts for Odoo is a lot of work, that can not be accomplished by a single party. We need your help. @@ -29,12 +41,3 @@ to upgrade a copy of your database and give us feedback. Thank you! -Contribute ----------- -In order to contribute to the OpenUpgrade project, please - -* Post your code contributions as pull requests on - https://github.com/OCA/OpenUpgrade -* Donate to the Odoo Community Association (https://github.com/sponsors/OCA) -* Hire any active contributor to this project to help you migrate your - database, and give back any code improvements developed during the project. diff --git a/docsource/analyse.rst b/docsource/analyse.rst deleted file mode 100644 index c72f94898090..000000000000 --- a/docsource/analyse.rst +++ /dev/null @@ -1,38 +0,0 @@ -How to run your own analysis -============================ -If you do need to run your own analysis, you need to perform the following -steps. - -* Set up two servers of subsequent Odoo releases. Make the following modules - available: `upgrade_analysis` and `openupgrade_scripts`. - - In the case of a release before 14.0, you need to run the fork from the - corresponding branch in https://github.com/OCA/OpenUpgrade. That fork also - contains `openupgrade_records`. - -* On both instances, install a database without demo data and - install the *upgrade_analysis* module, which is included in the - OpenUpgrade server distribution. This will add a menu - *Upgrade Analysis* to the Settings menu. - -* On both instances, install the modules that you need to write migration - scripts for, or use the *Install Modules* from the *Upgrade Analysis* menu. - -* On both instances: from the analysis menu, start the *Generate Records - Wizard*. - -* On the target instance (this is the more recent version): from the - analysis menu, select the *Comparison Configuration* option and - create a new config to connect to the other instance. In the form view of - this configuration record , start a *New Analysis*. - -The analysis files for each of the modules will be placed in the following -location: - -* In the case of Odoo modules: in the path that is indicated by the - `--upgrade-path` parameter of the Odoo server, or in the `scripts` - folder of the `openupgrade_scripts` module directory if it is available in - your addons path - -* In the case of OCA or custom modules: in the `migrations/` - directory in the module directory itself. diff --git a/docsource/analysis.rst b/docsource/analysis.rst deleted file mode 100644 index 23fc43713a61..000000000000 --- a/docsource/analysis.rst +++ /dev/null @@ -1,50 +0,0 @@ -Database analysis -+++++++++++++++++ - -Database analysis files are hosted in `scripts` subdirectory of the -openupgrade_scripts module directory in the https://github.com/OCA/OpenUpgrade -repository. - -You can create a custom set of analysis files on any set of modules between -any two versions of Odoo using the `upgrade_analysis` module that is hosted -in https://github.com/oca/server-tools. - -Earlier versions ----------------- - -In editions earlier than 14.0, the analysis files were hosted in the forks of -Odoo at https://github.com/OCA/OpenUpgrade. The module to create the analysis -was placed in each branch of the OpenUpgrade repository under -odoo/addons/openupgrade_records. - -Contents --------- - -The contents of the migration directory per module are: - -* A file `openupgrade_analysis.txt` that contains data model changes and - changes in the set of XML records -* A file `openupgrade_analysis_work.txt` which is a copy of the previous file, - annotated by the developers of the migration scripts -* A file `noupdate_changes.xml` containing changes that may need to be loaded - during module migration (after reviewing the contents of this automatically - generated file). -* A file `pre-migration.py` (or any Python file(s) starting with `pre-`) that - is executed by the Odoo migration manager just before the module is loaded. -* A file `post-migration.py` (or any Python file(s) starting with `post-`) that - is executed by the Odoo migration manager right after the module was loaded. -* A file `end-migration.py` (or any Python file(s) starting with `end-`) that - is executed by the Odoo migration manager when all modules have been - processed. - -The migration directory of the base module contains an additional file -`openupgrade_general_log.txt`. This file contains some statistics as well -as the analysis records of modules that could not be found in the target -release of Odoo anymore. - -.. toctree:: - :maxdepth: 2 - - analyse - xmlids - format diff --git a/docsource/coverage_legend.rst b/docsource/coverage_analysis/coverage_legend.rst similarity index 100% rename from docsource/coverage_legend.rst rename to docsource/coverage_analysis/coverage_legend.rst diff --git a/docsource/images/deleted.png b/docsource/coverage_analysis/images/deleted.png similarity index 100% rename from docsource/images/deleted.png rename to docsource/coverage_analysis/images/deleted.png diff --git a/docsource/images/new.png b/docsource/coverage_analysis/images/new.png similarity index 100% rename from docsource/images/new.png rename to docsource/coverage_analysis/images/new.png diff --git a/docsource/devfaq.rst b/docsource/devfaq.rst deleted file mode 100644 index 7593c61f9673..000000000000 --- a/docsource/devfaq.rst +++ /dev/null @@ -1,21 +0,0 @@ -Development FAQ -+++++++++++++++ - -How are new dependencies treated by the Odoo migration manager? - New dependencies (like the *edi* module is a new dependency of the - *account* module) will be detected by the upgrade process. The - Odoo server code is slightly modified to loop over this part - of the process to install new dependencies and then return to - upgrading the modules that depend on them, until no more modules - are processed. - -Are migration scripts fired when installing new modules? - Yes. That includes any new dependencies that the new version of any - module might declare. You might want to check for a non true value - of the *version* argument, or (better) make your script robust to - running against a database that it does not apply to, in anticipation - of any unknown unknowns. Also another argument for not running the - OpenUpgrade server in production, even though we both know that you - would never ever do so anyway. Developers are free to corrupt the regular - workings of Odoo if it helps the migration. For instance, from OpenUpgrade - 9.0 on, the workflow engine is disabled during field recomputation. diff --git a/docsource/index.rst b/docsource/index.rst index a178afd991eb..54c40dde8237 100644 --- a/docsource/index.rst +++ b/docsource/index.rst @@ -9,19 +9,20 @@ Welcome to OpenUpgrade's documentation! +++++++++++++++++++++++++++++++++++++++ -**If you just want to migrate your database and don't care about the technical details, refer to** :doc:`migration_details` - Contents: .. toctree:: - :maxdepth: 3 - - intro.rst - migrating_database.rst - community.rst - status.rst - analysis.rst - development.rst + :maxdepth: 2 + + 010_introduction.rst + 020_required_knowledge.rst + 030_coverage_analysis.rst + 040_run_migration.rst + 050_after_migration.rst + 060_odoo_migration_manager.rst + 070_migration_files.rst + 080_migration_script_development.rst + 090_contribute.rst You should find the latest version of this documentation here: ``_ diff --git a/docsource/migrating_database.rst b/docsource/migrating_database.rst deleted file mode 100644 index 2aaf8d68cf4f..000000000000 --- a/docsource/migrating_database.rst +++ /dev/null @@ -1,8 +0,0 @@ -Migrating your database -+++++++++++++++++++++++ - -.. toctree:: - :maxdepth: 2 - - migration_details - after_migration diff --git a/docsource/status.rst b/docsource/status.rst deleted file mode 100644 index ac07d24ec21e..000000000000 --- a/docsource/status.rst +++ /dev/null @@ -1,23 +0,0 @@ -Module coverage -=============== - -For all official addons, migration scripts should be provided for migrating -between subsequent major releases of Odoo. This is the goal of the project. -The design is such that the effort can be distributed between different -developing parties. See below for an overview of module coverage per version. - -.. toctree:: - :maxdepth: 1 - - modules150-160 - modules140-150 - modules130-140 - modules120-130 - modules110-120 - modules100-110 - modules90-100 - modules80-90 - modules70-80 - modules61-70 - modules60-61 - modules50-60 diff --git a/docsource/use_cases/value_mapping.rst b/docsource/use_cases/value_mapping.rst index 0a05aa684899..a41ab9f8482b 100644 --- a/docsource/use_cases/value_mapping.rst +++ b/docsource/use_cases/value_mapping.rst @@ -204,4 +204,4 @@ Notes This allows you to check that everything is correct after the migration, and to be used by further dependent modules in their migration scripts. Once this is done, you can delete this data. - (See: :doc:`../after_migration`) \ No newline at end of file + (See: :doc:`../050_after_migration`) \ No newline at end of file diff --git a/docsource/xmlids.rst b/docsource/xmlids.rst deleted file mode 100644 index f552d1b580e1..000000000000 --- a/docsource/xmlids.rst +++ /dev/null @@ -1,37 +0,0 @@ -XML IDs -======== -The OpenUpgrade analysis files give a representation of the XML IDs that a -module defines, in comparison with the previous release of the module. - -XML IDs which do not occur in the updated version of all installed modules -will be removed automatically by the Odoo server, if they do not have -the noupdate attribute. Therefore, you can ignore most entries here, such as - - * ir.actions.* - * ir.model.fields - * ir.model.access - * ir.model - * ir.ui.* - * res.country* - * res.currency* - -When XML ids of such record types change, the record will be recreated under -the new id, and the old record will be unlinked. - -More interesting are data that are defined with the noupdate flag. In general, -you need to cover two cases in your scripts. - -* Renamed XML ids of noupdate records. If a product category, or a res.groups - entry has been renamed, you should pass them to - :meth:`~openupgrade.rename_xmlids` - so that the item in your current database survives the migration in its - original form. If the XML id of a renamed noupdate record is not renamed - this way, the new record will be created in addition to the existing one, - and the existing record will lose its xml id. - -* Modified fields of noupdate records, as collected in the - `noupdate_changes.xml` in the migration analysis directory. When developing - the migration scripts for a particular module, review this file and comment - out any fields that must not be overwritten. If there is any data left that - you want to take, load it from your post migration script using - :meth:`~openupgrade.load_data`.