July 10, 2023

UserTesting and Sceneri: A Game-Changing Collaboration for User Experience

A conversation with Sceneri's CEO, Collin Bishop, on user-centric development

What companies has the team worked for in the past? What projects have you all been a part of?

Our company is composed of exceptional individuals who have honed their skills in game engine development within some of the industry’s largest studios. The team at Sceneri is comprised of industry veterans who have made significant contributions to renowned game development companies including Crytek, Nintendo, Massive, and Embark. Drawing upon this wealth of experience, we have acquired invaluable knowledge from our past endeavors, empowering us to navigate the game development landscape with remarkable efficiency and create a truly scalable game platform.

By embracing the power of modular plugins and harnessing the versatility of Vulkan portability, we have transcended the limitations of traditional codebases, enabling us to deliver exceptional gaming experiences across multiple platforms. This forward-thinking approach ensures that our creations are not only immersive and captivating but also accessible to a diverse and expansive audience.

Could you tell us about Sceneri?

Sceneri is a revolutionary game creation platform designed to empower a new generation of creators. It is a robust no-code application that enables users to play, create, and share games seamlessly across desktop and mobile devices, aiming to bring game creation to a wide casual audience worldwide. In essence, our mission is to modernize the traditional editor experience by extending its reach to as many devices as possible, including mobile platforms.

How did this story begin?

The inception of Sceneri traces back to the early 2000s modding scene of Crysis and the CRYENGINE Editor. It was during this time that modders discovered the immense potential of the toolset, which provided them with a comprehensive set of mechanics and materials on a AAA scale. Within the Crysis community, a diverse range of mods emerged, ranging from Crysis-based modifications to the full-fledged Mechwarrior: Living Legends adaptation.

MechWarrior: Living Legends, one of the many total conversion mods for Crysis (2007)

Inspired by the same passion and excitement that drove the modding community, our team is committed to delivering a similarly seamless and turnkey experience through Sceneri. We aim to provide creators with an intuitive and efficient game creation process while enhancing device portability and accessibility. Our goal is to empower creators to unleash their imagination and bring their unique visions to life, regardless of the platform they choose to create on.

What are the main strengths of your toolkit? Who is it meant for?

Technical strengths

  • Handheld and multi-platform: This is our key feature and helps separate us from other competitors in the game engine market.
  • No technical debt: building Sceneri from a “mobile-first” starting point means we have an advantage over bigger players in the industry.
  • Flexible and knowledgeable team: Engine development is a niche market, our team has broad and deep expertise across disciplines, from engine development to content creation and marketing.

User personas

  • Made for users of all levels: UGC is the bread and butter of our product, but we want to give newcomers and “only players” a great user experience too. Creators, remixers and players will find addictive
  • No coding skills required: newcomers can rejoice, utilizing the drag and drop mechanics within the asset browser for most standard game mechanics.

How and when did you decide to collaborate with UserTesting? What were your goals?

Right from the early stages of team construction, we engaged in rigorous deliberations on how to tackle the crucial aspect of quality assurance (QA). Drawing from our wealth of experience at Crytek and with CRYENGINE, we leveraged dedicated QA teams to ensure the highest standards of quality for both the engine and game development. This streamlined workflow was especially effective, given the similarities between console development and PC hardware across most players. However, we recognized that our expertise needed expansion, particularly in the realm of cloud streaming on mobile devices, where scalability became paramount.

As an ambitious startup, we pushed the boundaries of conventional thinking and sought ingenious solutions to meet the stringent demands of the mobile market. While exploring various beta testing options, we unearthed the commonly recommended tools such as ProductHunt, TestFlight, and Reddit communities for crowdsourcing testing—a testament to our thorough research and commitment to comprehensive QA coverage.

Ultimately, these forms of QA are fragmented and do not provide the level of testing we require and seek. While researching alongside these other options, the UserTesting platform was mentioned, and I was immediately captivated by the opportunity to conduct testing on a global scale.

How does UserTesting help you test across different devices and OS’s? You mentioned over 1000 different hardware and software configurations tracked. Could you elaborate?

Within the mobile development space, there are two distinct verticals: iOS and Android, which provide global coverage. Typically, you will seek to develop for either one of them. If you happen to choose Android, you will quickly understand that the volume of devices is much larger than you could ever test comprehensively. The same holds true for the iOS platform, where you may have fewer devices but still a volume greater than what in-house QA could ever cover.

Below, you will find some statistics from the initial month of our platform launch when we began utilizing UserTesting in the development environment. These results were obtained before the live release, providing us with the confidence that we were effectively engaging a broad range of testers and devices.

Analytics provided by: Incub8 Vortex

UserTesting provides a comprehensive testing environment encompassing both iOS and Android platforms, ensuring thorough coverage. As a company, we primarily focus on iOS initially, allowing us to conduct extensive testing on iOS devices, with a minimum model requirement of iPhone 7. Notably, when discussing specific issues, we can highlight the compatibility concern we encountered with iOS 15 and the virtual controller utilized within our games.

Please tell us about different groups of users. What did you learn? How would you describe your audience?

UserTesting offers a remarkable advantage in terms of user segmentation, allowing us to precisely target individuals based on their interests and geographic location. In our tests, we strategically divide users into different tiers, prioritizing those who actively engage in gaming. Furthermore, we specifically target users of both Roblox and Minecraft, two distinct platforms renowned for their exceptional world-building styles. When reviewing the surveys, we quickly found out how important the social and multiplayer aspects of the platforms were. Users of these two platforms almost treat them as primarily chat rooms with the ability to play or create experiences around that.

Having the ability to directly survey users empowers us to conduct focused tests that cater to the diverse preferences and creative abilities of our target audience. It allows us to know exactly what they desire, without relying on information disseminated by mass media.

Regarding our audience, we specifically target indie developers, a group well familiar to you at 80.lv. However, we also cater to a broader user base in the mobile creative market, with a particular emphasis on children and teenagers. It is worth noting that Unity boasts millions of users in the traditional engine market, and during the COVID-19 pandemic, Roblox experienced a peak monthly active user base of 200 million.

What would you say are the main benefits of collaborating with UserTesting? What main things did you learn?

UserTesting offers an exceptional advantage beyond its standard usage, as it provides us with a truly user-centric view on development. By analyzing recorded sessions of users actively engaging with our application, we make informed decisions that prioritize their needs and preferences. This approach has resonated strongly with prospective hires, with some talented individuals choosing to join our team specifically because of our commitment to user behavior and our dedication to optimizing the application by eliminating any points of friction. The value we place on user feedback and our unwavering focus on enhancing the user experience demonstrate our unwavering commitment to excellence.

Additionally, within the platform, we utilize some of their sentiment analysis and data summaries that accompany each testing run we initiate. Session summaries allow you to identify the most common issues and word clouds that showcase the mood and ease of use.

Overall, the collaboration with UserTesting has been invaluable, helping us identify and address issues such as tabbing crashes and audio distortion. The platform’s benefits extend beyond standard usage by providing sentiment analysis, session summaries, and data summaries that further enhance our understanding of user experiences. By employing UserTesting and leveraging its powerful features, we continue to refine and optimize Sceneri, ensuring that our game creation platform remains at the forefront of innovation and delivers exceptional experiences for creators and players alike.