Mosaic & ABSBackend when tiles are public

[jsga]

Hello! ✋🏿

Problem description

Titiler fails to fetch a mosaic.json stored in Azure Blob storage when the mosaic.json and the underlying COG tiles are public. In this case one would not need a key. These are the logs:

(...)
File "/opt/bitnami/python/lib/python3.11/site-packages/cogeo_mosaic/backends/__init__.py", line 40, in MosaicBackend
return ABSBackend(input, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<attrs generated init cogeo_mosaic.backends.az.ABSBackend>", line 14, in __init__
self.__attrs_post_init__()
File "/opt/bitnami/python/lib/python3.11/site-packages/cogeo_mosaic/backends/az.py", line 41, in __attrs_post_init__
HttpResponseError is not None
AssertionError: 'azure-identity' and 'azure-storage-blob' must be installed to use ABSBackend

I suspect this issue is related to developmentseed/cogeo-mosaic#191, let me know if I should post it in the cogeo-mosaic repository instead.

Expected Output

Titiler should be able to access the mosaic.json when public and no key is needed

[vincentsarago]

AssertionError: 'azure-identity' and 'azure-storage-blob' must be installed to use ABSBackend

azure blob storage for mosaic.json (via ABSBackend) needs optional dependencies: https://github.com/developmentseed/cogeo-mosaic/blob/main/pyproject.toml#L39-L41

you can fix this by installing azure-identity and azure-storage-blob in the environment you're running titiler

[jsga]

Thanks for answer so quickly! 🙌🏿

Installing those packages partially solve the problem. Maybe I can explain better the issue with a reproducible example, apologies for not doing it before.

I installed titiler version 0.10.2, to be precise, with this Dockerfile, adding the extra dependencies:

FROM ghcr.io/developmentseed/titiler-uvicorn:0.10.2
RUN pip install azure-identity azure-storage-blob
EXPOSE 8000
CMD uvicorn titiler.application.main:app --host 0.0.0.0 --port 8000

And use this mosaic.json which is public (you can access it with the browser):
https://mrvpublic.blob.core.windows.net/mrv/data/high-res/cragg/mosaic.json

docker build --tag titiler:latest .
docker run --name titiler -p 8000:8000 titiler:latest

Doing a GET to the COG endpoint works:

curl -X 'GET' \
  'http://localhost:8000/cog/info?url=https%3A%2F%2Fmrvpublic.blob.core.windows.net%2Fmrv%2Fdata%2Fhigh-res%2Fcragg%2F4.webp.tif' \
  -H 'accept: application/json'

{
  "bounds": [
    -2.1691387959734003,
    55.16654228629965,
    -2.1671800159737824,
    55.167545470233385
  ],
  "minzoom": 17,
  "maxzoom": 23,
  "band_metadata": [
    [
      "b1",
      {}
    ],
    [
      "b2",
      {}
    ],
    [
      "b3",
      {}
    ],
    [
      "b4",
      {}
    ]
  ],
  "band_descriptions": [
    [
      "b1",
      ""
    ],
    [
      "b2",
      ""
    ],
    [
      "b3",
      ""
    ],
    [
      "b4",
      ""
    ]
  ],
  "dtype": "uint8",
  "nodata_type": "Alpha",
  "colorinterp": [
    "red",
    "green",
    "blue",
    "alpha"
  ],
  "width": 10359,
  "count": 4,
  "height": 9243,
  "overviews": [
    2,
    4,
    8,
    16,
    32
  ],
  "driver": "GTiff"
}

But not to the mosaic, even though all files are public:

> curl -X 'GET' \
  'localhost:8000/mosaicjson/info?url=https%3A%2F%2Fmrvpublic.blob.core.windows.net%2Fmrv%2Fdata%2Fhigh-res%2Fcragg%2Fmosaic.json' \
  -H 'accept: application/json'

DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
        EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot.this issue.
        ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
        SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
        AzureCliCredential: Azure CLI not found on path
        AzurePowerShellCredential: PowerShell is not installed
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.

[vincentsarago]

I guess this is a question for @christoe who implemented the ABSBackend.

To me it's weird that need to use ABSBackend for public URL.

The issue is here: https://github.com/developmentseed/cogeo-mosaic/blob/main/cogeo_mosaic/backends/__init__.py#L38-L40, we are checking the url and if it has .blob.core.windows.net we use the ABSBackend backend, but for public URL we don't really need it 🤷

[christoe]

@jsga @vincentsarago : Sorry for the late reply. In our use case all files are hosted on non-public Azure Blob Storage, thus I never realised that this might be an issue. I guess we need to modify the backend selector logic. Question is how we tell public/non-public stores apart from the url..?

[vincentsarago]

I guess I was going for:

• az:// = private
• http:// = public

[christoe]

@vincentsarago : I just saw the issue in the cogeo-mosaic repo.. I'm a bit behind on the notifications, but I'll have a look at fixing this.