[Bugfix] DisableKVCache Context

[kylesayrs]

Purpose

• Fixes AttributeError: 'MllamaConfig' object has no attribute 'use_cache' #688
• Better catch models with unconventional config structure

Changes

• Use a no kv cache context which checks for use_cache and text_config.use_cache in cases like MllamaConfig

Testing

• Previously, attempting to forward pass meta-llama/Llama-3.2-11B-Vision-Instruct would lead to attribute error. Now runs normally with model.config.text_config.use_cache == False
• Regression tested cache disabling with meta-llama/Meta-Llama-3-8B-Instruct

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

[dsikka]

test?

[mgoin]

Could you try this on other models that doesn't have a use_cache in their config? Such as Llava https://huggingface.co/llava-hf/llava-1.5-7b-hf

[kylesayrs]

There are likely many ways to achieve this, including using the use_cache argument during the forward pass. However, this is known to at least work on the models we've tested

[kylesayrs]

Pending testing with llava models