Market research: Test audience survey

Part of this project involves evidence of rigorous market research to show that we have identified our target audience. This is an area which we as a team initially neglected, not intentionally, but given the fact that we are doing this degree on a part time basis, we had to give priority to creating a working prototype for a game first.

However, we have decided to rectify this by carrying out market research using what we have already made, similar to using a ‘test audience’ (Hoyt, A., n.d.). This practice would essentially be a form of blind play testing (What is Playtesting? – Definition from Techopedia, n.d.), in which we would get individuals who have no prior experience with our game to play through a level and record their experience.

The main things that we were looking to get from potential play testers was as follows:

  • How intuitive the game is and how easy it is to work out what to do
  • Is the game fun?
  • Is the game challenging?
  • Does the user want to see more of the game?
  • Does the level take a reasonable amount of time to complete?

To do this, we devised a survey in which participants were asked to first play a demo of the game. This demo consisted of the second level of the game, which is the level I designed and built. Participants would download and play the level from start to finish and then fill in the survey, which consisted of a series of questions about their experience playing the level.

You can find a link to the survey we created here.

Questions 

The first few questions that I asked were to ascertain the types of people who are filling in this survey. We were primarily, but not exclusively aiming this survey at fellow gamers who have a wide range of tastes as people who play games regularly will have a greater sense of what they look for in a game. These questions served the purpose of  ensure that we are getting a good representation of our target audience and are as follows:

  • What is your age range?
    • Under 18
    • 18-24    
    • 25-34    
    • 35-44    
    • 45-54    
    • 55-64    
    • 65+
  • How often do you play computer games?
    • Every day
    • A few times a week   
    • About once a week
    • A few times a month
    • Once a month
    • Less than once a month
    • Never
  • What type of games do you enjoy playing? (Please select all that apply)
    • Action
    • Adventure    
    • Puzzle    
    • RPGs    
    • Simulation    
    • Strategy    
    • Sports
    • Casual games
    • Not applicable / I do not play games

The second set of questions directly concerned the participants’ own experience of the game. These questions consist of a series of statements which the user has to indicate how strongly they agree or disagree with the statements. This was done using the Likert scale (Mcleod, 2019). They are as follows:

  • It was clear what I had to do when playing the test level.
  • The test level was both challenging and fun.
  • The game has an appealing and interesting concept.
  • The amount of time it took to complete the test level was reasonable.
  • After playing the test level, I want to know more about what the finished game would be like.
  • I will buy this game when it is released commercially.

In addition to the above, there will be an open ended question to ask if the participant has any further comments to add.  This will allow them to leave any additional feedback about the game demo that we may have overlooked and can use to our advantage.

Getting participants

We did not have Ipsos at our disposal, so it was not easy to circulate the survey far and wide, but we decided on several places online that we could share the survey online so that it would reach out to gaming communities, as well as people who play games regularly and also fellow game developers.

I shared the survey on a range of Discord servers, Facebook groups and subreddits, which were orientated around gaming, but also game development. I also shared with personal friends that I know are into gaming and asked them to also share.

Results

We ran the survey for a week and we got the following results

We got varied feedback, which I analysed collated and I then put my findings together to use in our pitch for Tuesday (Perry, L., Quinn, L., Tsaklev, G. and Waters, J., 2021).

Our findings

Demographics

From this data, we found that we got a good representation of our target audience as 86% of those who participated reported that they play games several times a week (61% of whom play games every day) and most enjoy playing games of a wide variety of genres.

While participants came from a range of age groups, the most common age group were those aged between 25-34, followed by those aged 18-24. This is to be expected as studies have been carried out in the United States that indicated that 38% of gamers are aged between 18 and 34, with the average age of gamers being 34 years old (Yanev, 2021).

Is it clear what the player needs to do?

Although we included some instructions in the survey, only 41% of participants reported that it was clear what they had to do when playing the level. A majority of participants responded that they neither agree, nor disagree with this statement indicating some confusions. This is of course amplified by the lack of a tutorial in our game demo.

When we finish this game, we will be sure to integrate a tutorial into that game, because while we do not want to offer the solution right away, it is important that we help the player to learn the mechanics so that they can enjoy and experience the game as best as possible.

Was the test level challenging and fun?

When choosing questions, I decided to include a question that asks whether the game is challenging AND fun, as opposed to having two separate questions for both. This was partly for the sake of brevity as we wanted to include as few questions as possible, but also because in our case, we wanted challenging and fun to go hand in hand with this game, whilst acknowledging they are two separate things.

Nevertheless this question proved to be divisive. 58% agreed that the game demo was challenging and fun. However, in the last question, which was an open question for further comments, several people indicated that the level was challenging, but not fun and vice versa. In hindsight, perhaps I should have asked these as separate questions.

Does the game have an appealing concept?

82% of the participants agreed that the game had an appealing and interesting concept – an amazing score for a game this early in development. This was echoed in many of the comments people left us in the last question too. Many enjoyed exploring the environment and solving the puzzle.

Was the amount of time that it took the level to complete reasonable?

72% agreed that the amount of time it took to complete the game was reasonable. However, in hindsight I should have deviated from the strongly agree to strongly disagree scale in favour of a scale which indicated whether it took too long, was too quick or just right, as this would have given us a better idea.

Did participants want more?

Not only did we run this survey to find out if participants enjoyed playing the demo, but more cruicially, we wanted to find out if they wanted to see more of the game based on what they had played. 79% who participated wanted to know what the finished game would be like and cruicially 55% indicated that they would buy the game when it is released commercially. Despite being in such an early stage, these figures are very promising and show that Construct could be a popular, fun and challenging game. With further iterations of this level, I expect that these numbers could increase.

Any other comments

The most telling question out of all of these was the last one which asked ‘Do you have any other comments that you wish to add about your experience playing the test level of Construct today?. This question was not quantitative like all the other questions within this survey, but the answers were incredibly telling and helpful.

I cannot go through every single answer one by one but many useful, encouraging and constructive comments were left in this answer, including:

  • Many really like the environment and enjoyed exploring
  • Several participants took ownership of the level and actually solved it in a different way to how I intended or planned
  • The metal rocks that are placed on the buttons are fiddly and may be better with a magnetic force of some sort
  • The level could do with a tutorial
  • The level is missing sound and a music track which would make the level more immersive
  • Most people expressed that they wanted to find out more about the game and they wanted to see more iterations of the level

The most surprising thing about this question was the amount of text that most participants left. I was only expecting it to be one or two sentences, if anything at all, but most participants left large paragraphs with detailed responses about the demo they had just played. It is an indication of how engaged all participants were with this level.

It is very encouraging that all of the people who took part, many of whom we did not know personally, were so willing to help us out and were willing to take the time out to play the game and engage with it meaningfully, while offering us constructive feedback to use.

If I was to do this again, I would most likely not just have the one text box at the end of the survey but perhaps I would also have an open question after each of the preceeding questions asking why they answered the way they did, This would make it easier to index all of the responses people gave about every aspect of the game.

References

Hoyt, A., n.d. How Audience Testing Works. [online] HowStuffWorks. Available at: <https://entertainment.howstuffworks.com/audience-testing.htm> [Accessed 11 August 2021].

Mcleod, S., 2019. Likert Scale Definition, Examples and Analysis | Simply Psychology. [online] Simplypsychology.org. Available at: <https://www.simplypsychology.org/likert-scale.html> [Accessed 14 August 2021].

Techopedia.com. n.d. What is Playtesting? – Definition from Techopedia. [online] Available at: <https://www.techopedia.com/definition/27197/playtesting> [Accessed 11 August 2021].

Yanev, V., 2021. Video Game Demographis – 27 Powerful Stats for 2021. [online] TechJury. Available at: <https://techjury.net/blog/video-game-demographics/> [Accessed 11 August 2021].

Perry, L., Quinn, L., Tenney, M., Tsaklev, G. and Waters, J., 2021. Construct – Demo level. The Cake is a Lie.

Perry, L., Quinn, L., Tsaklev, G. and Waters, J., 2021. Construct – Test Audience Survey. [online] Available at: <https://forms.office.com/Pages/AnalysisPage.aspx?id=s-4LVT1qRkahEfidAXd5LtBvCjt9AcVBuZ6C2l_rRY1URVlKN0xYS1BFUzFFRFg0NDUzU1NEQlM3TC4u&AnalyzerToken=Hw2REebTbPl3USUuV6gnAHnEWxrxylAV> [Accessed 13 August 2021].

2 Replies to “Market research: Test audience survey

Leave a Reply

Your email address will not be published. Required fields are marked *