Manifest serialisation from Python

[jdub]

Hello gang,

I looked through the docs but couldn't see anything that mentioned whether you expect migration runs to be repeatable.

i.e. if I do a run today and observe some failures, make some changes and do a run tomorrow, will it create all new items, or is it designed to replace existing items?

While I can't see any common GUIDs in POST/PUT calls between runs, I can see some overwrite=true query parameters that might suggest there's an expectation of repeatability.

Thanks,
Jeff

[dngtableau]

Hi Jeff,

Yes, it's designed to be repeatable. For multiple migration runs, you can pass in the previous manifest as a secondary argument to the execute method. Please see the python test app for additional details https://github.com/tableau/tableau-migration-sdk/blob/main/tests/Python.TestApplication/main.py.

Yes currently content types are expected to be replaced by subsequent runs. If you have questions about specific content types, feel free to ping this thread and we can answer in more detail.

Best,
Derek

[jdub]

@dngtableau great news, i wondered if the manifest stuff was relevant! i'll plug that in to my runner and see how i go – thanks for your help :-)

[jdub]

Hmm, do you have an example of serialising the manifest in Python, but outside the context of your test suite? It assumes it's running from a source checkout, and depends on things in the test suite, e.g. from Tableau.Migration.TestComponents.Engine.Manifest import MigrationManifestSerializer.

Also, are you aware of any funny interactions between filtering and manifests, e.g. if i set up a filter to only migrate e.g. 50 workbooks?

• run one: filter skips everything after 50 workbooks, saves manifest
• run two: loads manifest, skips (or checks) completed work, then the filter again skips everything after 50 workbooks, saves manifest (now showing 100 workbooks complete)
• and so on

Part of the reason I'm asking is that I'm not sure my simulations have ever successfully completed – I've literally never seen any of my post-migration logging. During a simulation last night, the Python process was using 29GB RAM.

So I'm going to do some experiments with filters, but I'm a bit in the dark without knowing how to serialise a manifest (so I can only viably run simulations), or how manifest+filters interaction.

[dngtableau]

Unfortunately we don't have an example yet. We're looking to add more features and documentations as we add more content types.

Can you share any logs and your migration plan above? The behavior above does seems odd. We would need more information to investigate the behavior.

[jdub]

I have now upgraded to 4.1.0, which is looking much nicer for Python! Still no manifest load/save examples, though it's easier to look at the resulting manifest object.

(I'm not on a fast enough network to run any full simulations, so I'll get back to you about the behaviour.)

[jdub]

I see 4.2.0 mentions manifest (de)serialisation in the release notes and even in the Python example! 👍🏻