I have always been fond of doing great projects involving some novel idea. The "novel" idea could be anything never done before or my sheer laziness towards literature survey. Anyways, here's some background.
In January 2013, I was "undernourished" with work. This was happening for the first time in last one and half years at grad school. I always had an urge to dirty my hands in the gaming field and hence I took up a directed research under a "gaming" professor.
I immediately teamed up with five others of my kind. It took us a couple of weeks to come up with an idea for the project and few weeks more to formulate it. "Let's make a racing game!", one said and everyone started pitching ideas. For some reason I was never satisfied with those ideas. "Racing games have been here for years! What's new in that?" I said to myself.
None of us had any experience with any sort of game development. Games involve knowledge of graphics, physics engine, state management, etc and these things vary with platforms. But still, I somehow convinced myself that if I am supposed to do something, it has to be awesome. Weeks passed and the abstract submission deadline came close. We looked into some of the past projects. Most of them involved creating weird zombie shooting games or the evergreen racing games. It then happened that one of my team mates had worked on a project which was sort of a supplement to games and it modified their behavior in real time. This interested me and initiated my thought process of what I can do on similar lines which was never done before.
After a couple of brain storming sessions with the team and some research on the Internet, we came up with an idea which looked very promising. We hadn't seen this in any of the games yet and were pretty convinced that this had been never done before. It was decided that we will be making a server API to be integrated into games.
Our abstract got approved as expected and we wrote the first piece of code. After a week or two, we met the professor to show our progress. Then emerged an interesting question, one which I felt, we should have raised ourselves - "Will it work?"
Like any enthusiastic developer we felt "Yeeaaahhh.. that's why we are doing it". Our implementation of the project would have been the proof of our concept. But the real challenge was not creating that system but evaluating whether it works. Any experienced computer geek could have made it in a couple of days or may be less.
"So guys, we need to provide an evaluation function. Any ideas?" I asked.
The thing with video games these days is that everyone plays them - from kids to elderly and even cats! They use various devices like laptops, mobile phones, gaming consoles, etc which made it mandatory for our idea to work across age groups and platforms. So how do we find out whether something is works for everyone?
"Surveys!!" - the good old method to evaluate whether something works or not.
Months passed by. The professor was happy that we had been thinking out-of-the-box and doing something novel. I was eager to finish the project at the earliest and relax during my last few weeks at the university. But were hit by a number of delays. Many of us had one or the other commitment which kept us occupied and our date for code freeze kept slipping.
We ultimately had to work overnight to finish the survey related code on the front-end. Two of us were coding java script with heavy eyes, so much so that we were unable to test our game for the tweaks that we had made. Finally, the game with the survey was working and the server was up and running. I decided to take an hour's nap before the long day started, with my partner already dozing off on the chair. His bent neck and open mouth facing the ceiling indicated he was half dead.
"Hello.. yeah am.. am coming in half an hour!" and I placed my phone aside as I tried to pull myself out of the bed. I reached the university after an hour. The table and chairs for the survey were set and pizza had just arrived. You do need to provide some incentive for people to take your survey. Since most of respondents would have been students we thought of having pizza to convince them.
After four eventful hours of surveying, we had failed! We had missed our mark by miles and here is why:
1. Bad location: The location we chose for the survey was perfect; but not in all aspects. It was the campus center and almost all the students, faculty and visitors passed though this location every day. The bad thing about it was that it was in the open without any power sockets nearby. This meant our laptops had to be recharged frequently at another location.
More importantly, the glare over the laptop screens made it harder for the players to understand and play the game. The soon-to-be summer sun made it even worse by pouring heat. This left us with unwilling and restless players.
2. Lack of convincing power: We were a bunch of sleep deprived developers, not marketers. We lacked the art of convincing people to play our short and sweet game and have free pizzas. It became more obvious when a kind person, from a nearby stall, offered to help us get more people to play our game. I don't know if I should have taken it but I was ashamed and chose to politely decline his offer.
I have always wanted to work on my people skills and this incident makes it a priority. My take - "Creating something awesome is one thing and making people realize it is awesome is something totally different".
3. Last minute coding: The last minute work on the code base not only left us sleep deprived, it also resulted in less time to test and debug the code. It turned out that the survey worked on some versions of the browsers but not on others. People had issues logging in to Facebook, which our application required. There were some more issues with Facebook API calls and our server API calls and it was a mess. To add to our miseries, the internet was extremely slow for most of the time during the survey, which meant it was testing people's patience along with ours.
In short, we spent our surveying time on debugging the issues we faced, looking for other "legal" areas to conduct the survey or thinking about alternative ways to gather legitimate data. In addition to this, we lacked a makeshift survey noting system.
We divided remaining cold pizza boxes among ourselves and headed home to figure out the issues. The shocker came when we realized that after spending so much time, money and energy on convincing those 20 odd people to play our game, their responses were not even stored on our back-end. Technically, we hadn't done the survey!
My Conclusion
A "people survey" is done over a product and not on a project. You need to fix most of the bugs and test it well on the systems you are going to use before you go out asking people to test it. People are the beta testers but beta testing comes after alpha testing.
People lack patience and time. You need to respect people for their time to have correct results. A working system is a prerequisite for that.
Analyze and evaluate the location of the survey beforehand. Number of people taking the survey was important but then availability of power and internet connection, as in our case, was more significant.
You need punch lines or slogans and make them heard. With positive attitude and confident charisma, you should convince them that they are interested in checking out what you have done.
And lastly, as a gentleman had noted - "You need a poster/banner saying what you are doing placed next to the desk".
Aftermath
After long debugging and testing session, we did convince ourselves that this is the best we could do for now. We shared the link among our friends and requested them to play our game. We still missed our mark but by a few feet and not miles. Our presentation along with the survey results did convince the people and the professor about feasibility and future scope of the project. It turns out we are quickly improving on the people skills. ;-)
Note: This is a true experience which I have tried to explain as a story. Please leave a comment.
Special thanks to my friend Aniket Vyas for his editorial review.
In January 2013, I was "undernourished" with work. This was happening for the first time in last one and half years at grad school. I always had an urge to dirty my hands in the gaming field and hence I took up a directed research under a "gaming" professor.
I immediately teamed up with five others of my kind. It took us a couple of weeks to come up with an idea for the project and few weeks more to formulate it. "Let's make a racing game!", one said and everyone started pitching ideas. For some reason I was never satisfied with those ideas. "Racing games have been here for years! What's new in that?" I said to myself.
None of us had any experience with any sort of game development. Games involve knowledge of graphics, physics engine, state management, etc and these things vary with platforms. But still, I somehow convinced myself that if I am supposed to do something, it has to be awesome. Weeks passed and the abstract submission deadline came close. We looked into some of the past projects. Most of them involved creating weird zombie shooting games or the evergreen racing games. It then happened that one of my team mates had worked on a project which was sort of a supplement to games and it modified their behavior in real time. This interested me and initiated my thought process of what I can do on similar lines which was never done before.
After a couple of brain storming sessions with the team and some research on the Internet, we came up with an idea which looked very promising. We hadn't seen this in any of the games yet and were pretty convinced that this had been never done before. It was decided that we will be making a server API to be integrated into games.
Our abstract got approved as expected and we wrote the first piece of code. After a week or two, we met the professor to show our progress. Then emerged an interesting question, one which I felt, we should have raised ourselves - "Will it work?"
Like any enthusiastic developer we felt "Yeeaaahhh.. that's why we are doing it". Our implementation of the project would have been the proof of our concept. But the real challenge was not creating that system but evaluating whether it works. Any experienced computer geek could have made it in a couple of days or may be less.
"So guys, we need to provide an evaluation function. Any ideas?" I asked.
The thing with video games these days is that everyone plays them - from kids to elderly and even cats! They use various devices like laptops, mobile phones, gaming consoles, etc which made it mandatory for our idea to work across age groups and platforms. So how do we find out whether something is works for everyone?
"Surveys!!" - the good old method to evaluate whether something works or not.
Months passed by. The professor was happy that we had been thinking out-of-the-box and doing something novel. I was eager to finish the project at the earliest and relax during my last few weeks at the university. But were hit by a number of delays. Many of us had one or the other commitment which kept us occupied and our date for code freeze kept slipping.
We ultimately had to work overnight to finish the survey related code on the front-end. Two of us were coding java script with heavy eyes, so much so that we were unable to test our game for the tweaks that we had made. Finally, the game with the survey was working and the server was up and running. I decided to take an hour's nap before the long day started, with my partner already dozing off on the chair. His bent neck and open mouth facing the ceiling indicated he was half dead.
"Hello.. yeah am.. am coming in half an hour!" and I placed my phone aside as I tried to pull myself out of the bed. I reached the university after an hour. The table and chairs for the survey were set and pizza had just arrived. You do need to provide some incentive for people to take your survey. Since most of respondents would have been students we thought of having pizza to convince them.
After four eventful hours of surveying, we had failed! We had missed our mark by miles and here is why:
1. Bad location: The location we chose for the survey was perfect; but not in all aspects. It was the campus center and almost all the students, faculty and visitors passed though this location every day. The bad thing about it was that it was in the open without any power sockets nearby. This meant our laptops had to be recharged frequently at another location.
More importantly, the glare over the laptop screens made it harder for the players to understand and play the game. The soon-to-be summer sun made it even worse by pouring heat. This left us with unwilling and restless players.
2. Lack of convincing power: We were a bunch of sleep deprived developers, not marketers. We lacked the art of convincing people to play our short and sweet game and have free pizzas. It became more obvious when a kind person, from a nearby stall, offered to help us get more people to play our game. I don't know if I should have taken it but I was ashamed and chose to politely decline his offer.
I have always wanted to work on my people skills and this incident makes it a priority. My take - "Creating something awesome is one thing and making people realize it is awesome is something totally different".
3. Last minute coding: The last minute work on the code base not only left us sleep deprived, it also resulted in less time to test and debug the code. It turned out that the survey worked on some versions of the browsers but not on others. People had issues logging in to Facebook, which our application required. There were some more issues with Facebook API calls and our server API calls and it was a mess. To add to our miseries, the internet was extremely slow for most of the time during the survey, which meant it was testing people's patience along with ours.
In short, we spent our surveying time on debugging the issues we faced, looking for other "legal" areas to conduct the survey or thinking about alternative ways to gather legitimate data. In addition to this, we lacked a makeshift survey noting system.
We divided remaining cold pizza boxes among ourselves and headed home to figure out the issues. The shocker came when we realized that after spending so much time, money and energy on convincing those 20 odd people to play our game, their responses were not even stored on our back-end. Technically, we hadn't done the survey!
My Conclusion
A "people survey" is done over a product and not on a project. You need to fix most of the bugs and test it well on the systems you are going to use before you go out asking people to test it. People are the beta testers but beta testing comes after alpha testing.
People lack patience and time. You need to respect people for their time to have correct results. A working system is a prerequisite for that.
Analyze and evaluate the location of the survey beforehand. Number of people taking the survey was important but then availability of power and internet connection, as in our case, was more significant.
You need punch lines or slogans and make them heard. With positive attitude and confident charisma, you should convince them that they are interested in checking out what you have done.
And lastly, as a gentleman had noted - "You need a poster/banner saying what you are doing placed next to the desk".
Aftermath
After long debugging and testing session, we did convince ourselves that this is the best we could do for now. We shared the link among our friends and requested them to play our game. We still missed our mark but by a few feet and not miles. Our presentation along with the survey results did convince the people and the professor about feasibility and future scope of the project. It turns out we are quickly improving on the people skills. ;-)
Note: This is a true experience which I have tried to explain as a story. Please leave a comment.
Special thanks to my friend Aniket Vyas for his editorial review.
