미드저니 프롬프팅 실험 - 레퍼런스 파라미터
Midjourney
요약
- 캐릭터 참조 매개변수는 참조된 이미지를 주요 주제나 대상으로 만든다.
- 캐릭터 참조시 참조 이미지에 뚜렷한 객체(ex. 인간, 동물 등)가 있다면, 미드저니 모델은 이를 주요 주제 및 대상으로 사용하고, 스타일과 색상 같은나머지 다른 요소들을 프롬프트 및 스타일 참조와 혼합한다.
- 뚜렷한 객체가 없다면, 모델은 프롬프트나 스타일 참조에서 일부 개념을 차용하면서 원본 참조 이미지의 구성을 근접하게 모방한다.
- 모델은 '배럴 디스토션'과 같이 이미지를 객관적으로 설명하는 문구를 처리하는 데 어려움을 겪는다.
- 모델이 이해할 수 없는 프롬프트의 경우, 배열과 구성 같은 요소를 적용하기 위해 스타일 매개변수를 사용해이미지를 직접 참조하는 것이 효과적이다.
- 캐릭터 참조 매개변수는 참조 이미지의 주제 및 객체를 강하제 보존하므로, 개념적인 부분이나 스타일적인 부분을 차용하려면 스타일 매개변수가 더 효과적이다.
미디엄에 작성한 원문
Experiment on Reference Parameter in Midjourney — cref, sref
Summary
The character reference parameter makes the referenced image the main theme or subject.
If the referenced image has a distinct figure, the model uses it as the main subject and mixes other elements like style and color with the prompt and style references.
Without a distinct object, the model closely mimics the original referenced image’s composition, borrowing some concepts from the prompt or style reference.
The model struggles with phrases that objectively describe the image, such as ‘barrel distortion’.
The character reference parameter strongly preserves the referenced image’s subject, so the style parameter is better for borrowing concepts without the subject.
Character Reference ( — cref)
Charater reference parameter enables the model to create images of the same character in the prameter.
(My interpretation) The parameter brings the subject (or alike) into an image generated, and it has a little impact on style and medium compared to style reference parameter.
Style Reference ( — sref)
Style reference parameter influences the style or aesthetic of images you want Midjourney to make
(My interpretation) Midjourney’s model borrows the style in the parameter — such as color, medium, composition — not subject in it. It has an impact overall but the subject.
References for the experiment & Characteristics
Each reference has a kind of converse color spectrum and dimensional expressions.
[A] Mondrian’s work: De Stijl, Cubism, Straight lines, Strict angles, Solid Colors
[B] A Fashion Photo: Black & White, Woman in wild eyelines, dramatic facial expression, Bold circles
Prompt
a perspective in highly barrel distortion, yellow and green, a woman gazing at the front without emotion — cref [REFERENCE 1]— sref [REFERENCE 2]— ar 16:9
Additionally, I wanted to know if a phrase in terms of perspective, arrangement and color when using reference parameters, so added ‘a perspective in highly barrel distortion’ which is also known as ‘fish-eye lens’ and ‘yellow and green’ phrase.
Result
Case 1: cref = Mondrian’s work, sref = A Fashion Photo
Case 1In regard to references
Mondrian’s work dominated almost all of the image from style to subject.
A Fashion photo’s existence of ‘woman’ and characteristic ‘circle pattern’ has mingled into the image, and they go hand in hand such in a way of the original composition.
In regard to prompt phrases
Both ‘barrel distortion’ and ‘green and yellow aren’t taken into account.
The expression of ‘gazing at the front’ is depected in only one image.
Case 2: cref = Mondrian’s work, sref = A Fashion Photo
Case 2In regard to references
The woman almost identical to A Fashion Photo became a main subject
Straight lines and strict lines from Mondrian’s work were adapted to its background, mixed with the circle pattern above the woman in the original Fashion Photo.
In regard to prompt phrases
‘Yellow and green’ is well applied.
‘Barrel distortion’ isn’t in place.
Takeaway & Interpretation
Character reference parameter has Midjourney model make an image whose the referenced image is the main theme or subject.
I speculate that if the character-referenced image has a distinctive figure, the model use it as main subject and mix the other substances in cref image — such as style, color, pattern — with prompt and style references.
If it has no distinguishable object in cref image, Midjourney model might strongly compose the image in the almost identical way of the original referenced image, partly borrowing some concepts in prompt or style reference
Again, like previous experiment, the model can’t properly finish the job related to phrase describing the image as an object such as ‘barrel distortion’ (objective description).
I really wanted to apply fish-eye lens style to the image, so I figured out how to realize it. According to the official document of Midjourney, it’s posslbe to use multiple refernce parameters, so I’ve got new famous reference.
Harry Styles!Prompt — sref 1 = A Fashion Photo, sref 2 = Harry Styles Album Cover
fish-eye lens photography, yellow and green background, a woman gazing at the front without emotion at the center — sref https://s.mj.run/YqHVLOMjQg8 https://s.mj.run/H0zVvl8yPBE
Result
Prompt — cref = A Fashion Photo, sref = Harry Styles Album Cover
fish-eye lens photography, yellow and green background, a woman gazing at the front without emotion at the center — cref https://s.mj.run/YqHVLOMjQg8 — sref https://s.mj.run/H0zVvl8yPBE
Result
Takeaway & Interpretation
A prompt which Midjourney’s model cannot interpret in an intelligible way should be referenced with style parameter to adapt the style such as arrangement and composition
Character reference parameter strongly maintains the subject in the referenced image, so if you want to just take some concepts of photo, it would be more effective to use style parameters.
