+- International Simulation Football League (https://forums.sim-football.com)
+-- Forum: Community (https://forums.sim-football.com/forumdisplay.php?fid=5)
+--- Forum: Media (https://forums.sim-football.com/forumdisplay.php?fid=37)
+---- Forum: Graded Articles (https://forums.sim-football.com/forumdisplay.php?fid=38)
+---- Thread: *In Favor of Draft Questionnaires (/showthread.php?tid=8833)
*In Favor of Draft Questionnaires - ExemplaryChad - 05-25-2018
This is not the first year that the start of scouting has meant some controversy among the league on draft preparation and procedure. In fact, the very topic of this piece has come up multiple times, and it never fails to generate lively discussion among players and front offices everywhere. I’ve been involved in some of these discussions, and while they’ve been contentious at times, no hard feelings have arisen in the aftermath (as far as I’m aware). The topic to which I’m referring, as can be gleaned from the title, is draft questionnaires for the purpose of scouting prospects.
Let’s first determine exactly what we’re talking about here. This is not a discussion of any general question and answer session between a prospect and an organization on an informal level. These are generally framed simply as conversations, and they have differing levels of both formality and usefulness for the parties involved. Rather, I’m talking about form questionnaires: a set of questions sent out by front offices to prospects that is the same across the board and requires prospects to answer without immediate feedback. Essentially, it is a survey, not a conversation. (This is not to say that it is not a dialogue, as you’ll see below.)
Now that we’ve got our terms squared away, let’s frame the argument. There are seemingly strong cases to make on either side of the discussion, and people seem equally passionate in either camp. Still, despite commonalities in fervor, there are fundamental disagreements in conclusions.
Those in favor tend to argue the following:
1a. Questionnaires are easier to send to multiple prospects at once.
2a. They offer a set of controlled conditions for prospects to respond to.
3a. They serve as a jumping off point for deeper conversations.
4a. They separate the dedicated from the inactive or disinterested.
Those opposed offer multiple claims as well:
1b. Questionnaires are impersonal and don’t make prospects feel special.
2b. They are too easy and show laziness in an organization.
3b. They do not demonstrate sufficient attention to detail in scouting.
4b. They are insulting to those with whom an organization is already familiar.
At face value, each list of claims seems like it could be correct. No doubt, those of you reading it are already choosing sides and preparing to go to battle for your favorite propositions. But hold your fire, comrades! I’m hoping I can do justice to either side while still attempting to persuade you of my own conclusion: that questionnaires are more useful than harmful.
I do believe that questionnaires offer an easy, controlled, dichotomous way to discern the intentions and aptitudes of a draft prospect. As this is my position, I will attempt to justify these claims and address potential objections. Let’s examine more closely.
1a. The first claim, that questionnaires are easier than personalized communiques, is mostly irrefutable. Writing one list of questions and sending it out to everyone is certainly easier than tailoring questions to each individual prospect, especially those about whom very little is known. The question is whether it’s too easy, demonstrating a lack of motivation or dedication to the process. I’ll talk more about that later.
2a. The second claim is that questionnaires offer a controlled environment from which to gather responses. This claim is true as well. If I’m interviewing candidates, and I ask one which color is their favorite and the other which ice cream they prefer, how am I to compare the two candidates? Similarly, if I’m only having informal discussions with each prospect, I can get a sense of what kind of person they are going to be in the locker room, perhaps, but I’m not going to get an equal measure of each individual. What happens if I talk to one person on a bad day and another person on a good one? That will now color my perceptions of each prospect unfairly. With a carefully crafted questionnaire, the sender allows prospects to take their time in responding and be as thoughtful (or as thoughtless) as they intend to be.
3a. The third claim posits that a questionnaire can serve as a starting point for more individual conversations. This is an important point. Many opponents of questionnaires assume that the survey itself is the beginning and end of the scouting process. If a front office is doing its job, this is not the case at all. Instead, it allows a front office to ask questions expanding upon the answers of the prospect and have a conversation in a more directed and deliberate way. This makes it so that the front office and the prospect are not just “shooting the shit,” as the saying goes. They are communicating efficiently and effectively, necessary components in a compressed and thorough draft process such as the one we use here in the NSFL.
4a. The final claim of questionnaire proponents is that they separate dedicated and active members from those who are neither. This is perhaps the weakest claim of the bunch, but it does have some validity. There are other explanations for prospects who don’t respond to questionnaires, to be sure. They could simply not get around to it; they may not be familiar enough with the site to notice that they have it; they may be protesting the process altogether and not answer because of that (though in the eyes of some front offices, this would be a red flag in itself); or they may be inactive or planning to be so. This last category is the one worth sussing out. While a prospect should not be dismissed out of hand for failing to respond to a questionnaire, it can count as a strike against a borderline-active player. It can demonstrate a lack of activity, and as we all know, that can be an extremely valuable bit of information.
Let’s now move on to the claims against the usefulness of questionnaires and see if we can refute some.
1b. The most common claim against the use of these types of surveys is probably this first one. They don’t make players feel special. This is certainly true for some people and is crucial in a number of ways. The player experience is the thing that should come first, to be sure. We are a players’ league, and we want maximum engagement and maximum enjoyment for everyone involved -- GMs, Co-GMs, Head Office, and every single other player on the site. So to not make players feel special with our recruiting process is certainly something we should be leary of. However, does this necessitate the conclusion then that every action taken, at all times, should be maximally tailored to a player’s individual experience? After all, these players should be made to feel special at a number of key points before they are ever drafted at all: Recruitment (by the recruitment team), character creation (by the rookie mentor and approval teams), DSFL team contacts upon first joining a team (GMs and Co-GMs), and NSFL teams as well. There exists a myriad of opportunities to make players feel special throughout this entire process, up to and including the scouting and drafting of players. While a more personal touch is probably needed at some point during the scouting process, must it be at every single turn? If a player balks at a single step in the long process of joining and playing in this league not being perfectly tailored to their experience, doesn’t that say more about the player than the league? It’s simply not realistic to expect every part of the process to perfectly address every individual need of every individual player. It’s literally an impossible endeavor. While some are turned off by this sort of questionnaire, some absolutely enjoy it and recognize its value. (For me, I was extremely excited to get a list of questions where I could fully express myself and make an impression on interested organizations.) This doesn’t even mention the fact that follow-up communication absolutely should be the norm after sending out a form questionnaire anyway. Ultimately, player experience and engagement is absolutely the number one priority in this league, as it should be. But one single instance that prioritizes efficiency and effectiveness over personal experience should not, and hopefully does not, break the entire league for a player, especially when considering the fact that an efficient, effective organization is more appealing to most players than the alternative anyway.
2b. Do form questionnaires make an organization lazy? It certainly could appear so given point number one in favor of questionnaires -- specifically, that they are easier than a different form of communication. But I can guarantee from experience in my team’s scouting organization that this is far from the be-all and end-all of the drafting and scouting process. It may look like taking the easy way out when it comes to contacting prospects, but I would argue that it is not. It can make information easier to organize and compare; this is absolutely true. But it is not some easy process that is just a “set it and forget it” type of endeavor. First of all, it takes a ton of forethought and insight in order to craft a well-made survey. You want to be comprehensive without being excessive. You want to find out if the player is a good fit without coming across as adversarial or unfeeling. You want to make sure you’re communicating information as well as receiving it. A good questionnaire is a dialogue, not just a list of data points (though it does obviously contain plenty of data). And, of course, there is always the issue of follow-up. If a team follows up with prospects after getting some answers back, they are being far from lazy in most cases.
3b. I have heard the claim made that a team who sends out questionnaires is not paying enough attention to the scouting process or the prospects themselves. For instance, if a prospect is a GM in another sim league, they should not have to fill out a Q and A because we should already know who they are and what they’re about. This argument has some flaws. First of all, perfect knowledge is impossible. Some teams will have an advantage in this regard -- teams with plenty of players in other leagues or the otherwise necessary connections to get this information -- but it is simply not feasible for every team to know this at every turn. Second, just because a player is involved in other leagues or has some other detail that could help determine their worth as a prospect, it does not necessarily translate to success in the NSFL. Just by virtue of this being a football sim means that players who have different attitudes towards the sport will have differing levels of interest in the league. Interest in hockey does not always equal interest in football. Third, A GM in another league might be devoting so much time to it that they have little left to be active in this league. The information may not translate as cleanly as the prospect would like. Finally, a team missing a detail does not mean that the team habitually misses details or that there are somehow flaws in the organization. Like I said, perfect knowledge is impossible. It doesn’t matter how good a team’s process is; there are bound to be things that get missed. It’s a shame, but it’s undeniably true.
4b. Draft questionnaires as an insult is a stronger reaction than I ever expected to see in these conversations, but see it I have. To this, I would like to first say that it is certainly never intended to insult anyone, and I offer my apologies on behalf of any hurt feelings where I have been involved regarding scouting questionnaires. A questionnaire is not an indictment that we don’t know who you are or that we don’t like you. It is merely a means of gathering and dispersing more information. Having a friend is very different from wanting that friend to work with you. It’s the same with hobbies. There is certain information that I’m going to want about you before we decide to undertake this game together, assuming I want to be successful at the game in question. I have many friends that I enjoy being around, but I wouldn’t want to work with them. It’s the same with playing role-playing games or seeing movies or writing a collaborative work of fiction. This is also certainly true in the inverse. I’m sure many of my friends wouldn’t want to involve me in certain activities that they themselves enjoy (e.g. hunting or going to a Brazilian steakhouse or playing baseball). We have to see if we would be a good fit for this particular environment. It says nothing about your character or mine. It’s just a matter of fit.
Obviously, I’ve taken more time on the con side of the argument than the pro. My goal is only to refute claims that seem spurious while supporting those that seem salient to me. Since I already agree with the arguments in favor, there is less need for me to spend time refuting them. With the arguments against, I have to explain and refute, rather than just explain. I hope that in doing so, I’ve given at least some understanding, if not actually changed any minds.
For those of you who couldn’t be bothered to read the whole thing above, here’s the tl;dr. Form questionnaires are good for teams and, when done well, for prospects. While they may seem impersonal and lazy, they are neither of those things and, in fact, serve as a very helpful tool for conducting deeper scouting operations and getting an initial picture of candidates.
So, if you’re a prospect thinking about ignoring those initial questionnaires because you don’t like the idea, I hope this will give you pause. I hope you’ll give a team the chance to engage in deeper discussion. I hope you’ll realize that this is not the only way your worth is being determined, that it is only one measure of about a thousand that go into a team’s decision making. And, finally, I hope you dance.
I’m sure there are reasons that I’ve missed on both sides of the aisle that could sway someone one way or another. I’m also sure that there will be problems with my arguments that people would love to point out to me. I would love to further discuss either of these things, and more, below! Thank you for your attention on this matter, and I look forward to a lively discussion in the coming days/months/years.
Word count: 2,449
*In Favor of Draft Questionnaires - bovovovo - 05-25-2018
I DISAGREE
EDIT: okay now that I've actually read it here's what I think...
I think questionnaire's are perfectly fine and agree with all your points as to why. But I also think that individual conversations are inherently better. You learn more about the prospect, the prospect learns more about you, it's more fun (and that's what we're all about yeah?), and it gives prospects a chance to individualize themselves more than a generic questionnaire allows them to.
And even though *I* think questionnaire's are fine, it's become pretty obvious that many prospects don't like them. At best they're annoyed by it and at worst they're offended by it. I think as a GM it's better to put your best foot forward and knowing that there's prospects out there who don't like questionnaire's is enough for me personally to just never bother with them.
I kinda like having those conversations too lol
But overall, really good article that raises good points
*In Favor of Draft Questionnaires - Durden - 05-25-2018
Wall of text crits you for 728 damage.
*In Favor of Draft Questionnaires - ExemplaryChad - 05-25-2018
(05-25-2018, 07:57 AM)Durden Wrote:Wall of text crits you for 728 damage.
I think you mean 2,449 damage.
*In Favor of Draft Questionnaires - AdamS - 05-25-2018
_ _ _ _ _ _ _
_ _ _ _
Noun
*In Favor of Draft Questionnaires - ExemplaryChad - 05-25-2018
(05-25-2018, 07:49 AM)bovovovo Wrote:And even though *I* think questionnaire's are fine, it's become pretty obvious that many prospects don't like them. At best they're annoyed by it and at worse they're offended by it.
Yeah, I think that's a shame. I guess the article is really aimed at prospects to see why we use them, really. I'd hate to continue doing something that some people don't like. At the same time, I know there are people (like me) who really like the flexibility and deliberateness of them as well.
Basically, I think all prospects should agree with me.
*shrug*
*In Favor of Draft Questionnaires - Durden - 05-25-2018
If someone is drafting me, I want it to be personal.
I suppose if I was semi-active, I wouldn't care about doing an interested prospect thread or a questionnaire, but if I'm a high pick, you better talk to me. As seen in this last draft.
*In Favor of Draft Questionnaires - timeconsumer - 05-25-2018
We should have one thread with a set of questions all GMs have agreed upon and post it for all prospects to fill out. That way you still get your basic questions handled, but the prospect doesn't end up answering a bunch of the same questions 8 times.
*In Favor of Draft Questionnaires - PDXBaller - 05-25-2018
(05-25-2018, 05:16 AM)timeconsumer Wrote:We should have one thread with a set of questions all GMs have agreed upon and post it for all prospects to fill out. That way you still get your basic questions handled, but the prospect doesn't end up answering a bunch of the same questions 8 times.
This. It was annoying for us to fill out the SHL IP Threads because over half of the questions were the same.
*In Favor of Draft Questionnaires - bovovovo - 05-25-2018
(05-25-2018, 08:16 AM)timeconsumer Wrote:We should have one thread with a set of questions all GMs have agreed upon and post it for all prospects to fill out. That way you still get your basic questions handled, but the prospect doesn't end up answering a bunch of the same questions 8 times.
Not a bad idea actually