Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

modify run_benchmark.sh #75

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

modify run_benchmark.sh #75

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a27e980
modify run_benchmark.sh
smile0655 Apr 24, 2023
5f274f8
modify run_benchmark.sh
smile0655 Apr 24, 2023
6e087f7
refine
smile0655 May 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 33 additions & 11 deletions onebench/diffusers/run_benchmark.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,13 @@ set -ex
# Model download
# user_header="Authorization: Bearer xxxxxx"
# wget --header="${user_header}" https://huggingface.co/CompVis/stable-diffusion-v-1-4-original/resolve/main/sd-v1-4.ckpt -O models/sd-v1-4.ckpt
STABLE_VERSION=${1:-"sdv1_5"} # sdv1_4 sdv2 sdv2_1 taiyi

STABLE_VERSION=${1:-"sdv1_5"} # sdv1_5 sdv2_0 sdv2_1 taiyi
INSTALL_ONEFLOW=${2:-"master"}
CUDA_VERSION=${3:-"cu116"}
CUDA_VERSION=${3:-"cu117"}

export HUGGING_FACE_HUB_TOKEN=hf_
export HF_HOME=/hf/home
export HF_HOME=/data/home/zhouhongjun/sd_test/hf/home
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里还是用原来的，不要出现个人信息在里面

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我修改了下，改成下面这种可以自动创建文件夹的方式，因为原来的地址的话会有pemisson denied的提醒。

SRC_DIR=$(realpath $(dirname $0))
mkdir -p ${SRC_DIR}/hf/home
export HUGGING_FACE_HUB_TOKEN=hf_
export HF_HOME=${SRC_DIR}/hf/home


# install oneflow
python3 -m pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
Expand Down Expand Up @@ -72,17 +73,36 @@ echo "oneflow-diffusers(git_commit)=$ONEFLOW_DIFFUSERS_COMMIT"
python3 -m pip install -e .[oneflow]
cd ..

if [ "$STABLE_VERSION" == "sdv2_0" ]; then
sed -i '/import oneflow as torch/a\torch.mock_torch.enable()' ./scripts/$BENCHMARK_SCRIPT
sed -i '/from diffusers import (/,/)/c\
from diffusers import EulerDiscreteScheduler\' ./scripts/$BENCHMARK_SCRIPT
sed -i '/from diffusers/a\from onediff import OneFlowStableDiffusionPipeline as StableDiffusionPipeline' ./scripts/$BENCHMARK_SCRIPT
fi

if [ "$STABLE_VERSION" == "sdv1_5" ]; then
sed -i '/import oneflow as torch/a\torch.mock_torch.enable()' ./scripts/$BENCHMARK_SCRIPT
sed -i 's/from diffusers/from onediff/g' ./scripts/$BENCHMARK_SCRIPT
fi

if [ "$STABLE_VERSION" == "taiyi" ]; then
sed -i 's/from diffusers/from onediff/g' ./scripts/$BENCHMARK_SCRIPT
fi

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这三个判断有没有可能优化一下，比如from diffusers/from onediff是不是公用的了。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

关于这三个判断主要是因为diffusion-benchmark仓库的原始文件的import写的格式不一样。stable_diffusion_2.py要从

from diffusers import (
    OneFlowStableDiffusionPipeline as StableDiffusionPipeline,
    OneFlowEulerDiscreteScheduler as EulerDiscreteScheduler,
)

改成

from diffusers import EulerDiscreteScheduler
from onediff import OneFlowStableDiffusionPipelinev as StableDiffusionPipeline

stable_diffusion_v1_5.py和taiyi_stable_diffusion_chinese.py要从
from diffusers import OneFlowStableDiffusionPipeline as StableDiffusionPipeline
改成
from onediff import OneFlowStableDiffusionPipelinev as StableDiffusionPipeline

最后stable_diffusion_v1_5.py和stable_diffusion_2_0.py 要单独添加
torch.mock_torch.enable()

所以感觉上找不到三个通用的修改方法。

sed -i '/for r in range(repeat):/a\
cmd = "nvidia-smi --query-gpu=timestamp,name,driver_version,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used --format=csv" \
os.system(cmd)' ./scripts/$BENCHMARK_SCRIPT
# if [ "$STABLE_VERSION" == "sdv1_5" ]; then
# sed -i 's/cmd = "nvidia-smi/ &/' ./scripts/$BENCHMARK_SCRIPT
# sed -i 's/os.system(cmd)/ &/' ./scripts/$BENCHMARK_SCRIPT
# fi

if [ ! -d "./diffusers" ]; then
git clone --depth 1 https://github.com/Oneflow-Inc/diffusers.git
if [ "$STABLE_VERSION" == "sdv1_5" ] || [ "$STABLE_VERSION" == "taiyi" ]; then
sed -i '/@click.option("--output", default="output")/a\@click.option("--height", default=512)\
@click.option("--width", default=512)' ./scripts/$BENCHMARK_SCRIPT
sed -i 's/images = pipe(prompt).images/images = pipe(prompt, height=height, width=width).images/g' ./scripts/$BENCHMARK_SCRIPT
sed -i 's/def benchmark(token, prompt, repeat, output):/def benchmark(token, prompt, repeat, output, height, width):/g' ./scripts/$BENCHMARK_SCRIPT
fi

cat ./scripts/$BENCHMARK_SCRIPT
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是没用

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是为了查看修改是否成功的，可以删掉



GPU_NAME="$(nvidia-smi -i 0 --query-gpu=gpu_name --format=csv,noheader)"
GPU_NAME="${GPU_NAME// /_}"

Expand Down Expand Up @@ -119,14 +139,16 @@ $DL_FRAME 2>&1 | tee ${LOG_FILENAME}.log
echo "oneflow-version(git_commit)=$ONEFLOW_VERSION" >> ${LOG_FILENAME}.log
echo "oneflow-commit(git_commit)=$ONEFLOW_COMMIT" >> ${LOG_FILENAME}.log
echo "diffusion-benchmark(git_commit)=$DIFFUSION_BENCHMARK_COMMIT" >> ${LOG_FILENAME}.log
echo "oneflow-transformers(git_commit)=$ONEFLOW_TRANSFORMERS_COMMIT" >> ${LOG_FILENAME}.log
#echo "oneflow-transformers(git_commit)=$ONEFLOW_TRANSFORMERS_COMMIT" >> ${LOG_FILENAME}.log
echo "oneflow-diffusers(git_commit)=$ONEFLOW_DIFFUSERS_COMMIT" >> ${LOG_FILENAME}.log

### pytorch
DL_FRAME="pytorch"
LOG_FOLDER=stable_logs/$GPU_NAME/$DL_FRAME
sed -i 's/oneflow as //g' ./scripts/$BENCHMARK_SCRIPT
sed -i 's/OneFlow//g' ./scripts/$BENCHMARK_SCRIPT
sed -i 's/torch.mock_torch.enable()//g' ./scripts/$BENCHMARK_SCRIPT
sed -i 's/from onediff import OneFlowStableDiffusionPipeline/from diffusers import StableDiffusionPipeline/g' ./scripts/$BENCHMARK_SCRIPT
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这一步看看git checkout可以做到不

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果这里使用git checkout的话，要再添加一句计算显存的

sed -i '/for r in range(repeat):/a\
        cmd = "nvidia-smi --query-gpu=timestamp,name,driver_version,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used --format=csv" \
        os.system(cmd)' ./scripts/$BENCHMARK_SCRIPT

那我采用这个修改方式吧

cat ./scripts/$BENCHMARK_SCRIPT

TORCH_VERSION=$(python3 -c 'import torch; print(torch.__version__)')
python3 -m pip uninstall -y diffusers
Expand Down