I'm going to try doing a lora for moss-image-sft with my danbooru 40k dataset using the following slopbot-recommended settings:
train:
batch_size: 1
cache_text_embeddings: true
steps: 10000
gradient_accumulation: 1
train_unet: true
train_text_encoder: false
gradient_checkpointing: true
noise_scheduler: "flowmatch"
optimizer: "adamw8bit"
lr: 1e-4
lr_scheduler: constant
dtype: bf16
model:
name_or_path: "OpenMOSS-Team/MOSS-Image-SFT"
arch: "moss_image"
quantize: false
quantize_te: false
low_vram: false
I'm not sure how many steps it's going to actually need, but I'll be watching test prompts to see how anatomy or pose-adherence changes.