Would you mind elaborating on “control the context to eliminate bias and gaming” under this situation?
Sure, apologies if you already know any of this.
As with other scientific fields, there are guidelines and processes in place to evaluate the structure and approach for research.
iirc you don’t technically have to adhere to them, but it will certainly be a point of industry and peer criticism if you don’t, sometimes leading to papers not being accepted for journals and other more esoteric consequences.
This is one of the reasons proper peer review is important.
A basic example would be picking from (or narrowing to) an appropriate subset of the population.
If you were trying to perform research with the goal of evaluating the population as a whole, running your experiment exclusively with women between the ages of 18-25 would immediately be picked up as a reason the results can’t be trusted (in terms of the stated goal).
A slightly less obvious example (for certain kinds of experiments) would be sentence structure and unconscious bias through contextual information.
When wording questions and examples it is easy to introduce a bias in the tone and word choice, which can affect the outcome of the research.
A simplistic summary is that there is a bias (unconscious or otherwise) against people with “ethnic” sounding names on their resume.
There is, of course, more nuance to it than that, but still.
This is much less cut and dry because sometimes the bias is the thing being studied and forms a part of the test, which is why when creating these kinds of experiments the process is carefully evaluated and revised, hopefully by multiple people.
Another one you touched upon already is context, the time of day, life events, general disposition etc.
Good test design will try to account for as much of this as possible (though it’s unlikely to remove it all entirely).
Obviously the more questions asked, the more granular the results can become, so I’ll grant that.
That’s not always strictly true, quality is also important and there are diminishing returns on quantity, the length of a questionnaire can sometimes have it’s own effect on the results for instance.
This relates to your final point: What would I consider to be the test’s objectives? For me, it’s an exercise in gleaning insight into one’s own personality; to help with reflection and introspection. To identify your strengths and weaknesses. In some sense, to provide some identity. I can’t tell you how I felt understood. I actually teared up while reading the analysis for the first time. As something of an outsider for much of my life it was like it filled in the missing pieces I long suspected and yet always doubted. Like I said I can’t speak for what others got out of the test, but it was the best therapy I ever received. (And for context, I read every other generalized group to make sure it wasn’t generalized astrological bullshit where every description could match every person, for which nothing came close).
It sounds like this experience was/is of great use to you. I’ve heard similar things about ADHD and ASD diagnoses.
Finding your tribe/place sounds great.
What i would say is that people who don’t have this level of resonance with the results could well see it less favourably than you.
That isn’t necessarily because they performed the test (or interpreted the result) incorrectly, it could just mean less to them.