How AI Is Changing Games?

AI Now Reads User Feedback Before You Do

“I get that AI makes things easier… but does it mean we have to change the way we work?”

This question is becoming increasingly common among game service stakeholders. And the answer is: in many ways, the change has already begun

ℹ️ AI as a Service Tool, Not Just Technology

Many game planners, project managers (PMs), and game operation teams still think of AI as something designed for engineers or designers.

However, the real value of AI lies not in what it is, but in how and where it’s used.

While early examples were mostly technical, today we’re seeing more practical cases where AI directly helps business and service teams.

Especially in live operations, where understanding players and responding to feedback quickly is everything, AI is proving to be a practical and powerful ally.

ℹ️ Summarizing User Feedback: Find the Real Issues, Faster

If you’ve ever overseen a game launch or update, you’ve likely faced situations like these:

“I thought after game launch would be smooth… but now we’ve got thousands of App Store reviews. The dev team wants a summary of the issues, and I don’t even know where to start.”

“Support tickets are piling up. It feels like there’s a common issue here, but I can’t put my finger on it.”

“We ran FGI sessions and user interviews ourselves, but there’s no time to sort through all the feedback.”

This is where AI can step in to help:

“Sentiment analysis” of thousands of app store reviews

Automatic “summarization” and “key phrase extraction” from user interview transcripts

Classification of feedback into complaints, suggestions, and praise using large language models (LLMs) with tools like “LangChain and RAG”

There’s a small technical point worth noting here. If you’ve attended any recent AI seminars, you may have heard terms like LangChain and RAG. They might have seemed clear at the time but can be difficult to explain later.

The following summary may help you understand and explain them more easily

* What is LangChain?

LangChain is like an automated workflow system for AI. Think of it like this:

When someone asks you to write a proposal, you probably go through these steps:

① Research → ② Summarize → ③ Draft outline →

④ Format as a document

LangChain allows AI to handle those steps in sequence by itself.

* Also, what is (Retrieval-Augmented Generation)?

RAG means showing AI the right documents first before asking it a question.

Normally, AI only answers based on what it has memorized. With RAG, you give it the exact user reviews or documents to read before generating a response.

In short, it’s a way of making sure AI doesn’t generate vague or irrelevant answers. You provide the necessary reference materials first and then instruct the AI to “answer only based on this information.”

For example, if you ask, “Why are players so upset with our game?”

A typical AI might respond with something general like, “There could be various reasons, such as aggressive monetization or lack of communication.”

However, if you apply the RAG approach and provide actual review data as reference, the AI can respond more specifically:

“About 70% of user reviews repeatedly mention issues with confusing package options and an unfriendly tutorial.”

ℹ️Real-World Use: Joycity’s VOC System

Joycity presented a great case study at the conference last year, showing how their live ops and data teams built an AI-driven Voice of Customer (VOC) system.

The core of this system is a structure where AI automatically analyzes posts from user communities and support inquiries, allowing the operations team to review and respond immediately.

The following key functions were implemented using AI during this process:

Daily post volume: Automatically tracks the number of new users posts each day and compares it to the previous day

User feedback (VOC) summaries: Generates daily summaries of updated VOC data using generative AI

Sentiment analysis: Visualizes a five-level emotion score using a custom-trained LLM model

Keyword extraction: Identifies key issues from feedback and highlights the top five with visual summaries

Multilingual trend tracking: Uses NLP models to analyze global user community trends across multiple languages

A screenshot of Joycity’s VOC analysis system, providing a real-time overview of user feedback brief

Joycity also applied the RAG method, enabling the AI to reference actual user review content directly. By using advanced LLMs like GPT-4 and Claude, they generated summaries that were both accurate and contextually relevant.

One notable approach was their use of “temporary labeling.”

Instead of manually categorizing all the data, they labeled only a small portion. The AI then selected and learned from the data it was most confident in. This led to the development of an automatic classification model with approximately 89% accuracy.

What does 89% accuracy mean in this case? It means that in 9 out of 10 cases, the AI’s classification matched a human judgment. For a task as subjective as sentiment analysis, exceeding 85% is a meaningful result.

This allowed Joycity’s live ops and service teams to extract insights more quickly, without manual sorting and focusing on the most urgent issues.

ℹ️ Expanding the VOC System: “AI Summarizes, Humans Decide”

Joycty plans to further evolve this VOC system into a broader decision-support tool, with the following improvements:

Visualizing trends over time by mapping user feedback against events and patch timelines

Combining insights from user sentiment with metrics like revenue and player level data

Introducing lightweight LLMs such as Gemma, LLaMA, and Phi to improve efficiency

Automatically generating AI-powered trend reports to support faster internal communication

While these may sound technical at first, the idea is simple:

“Let AI handle the summaries, so game teams can focus on making decisions and taking action.”

This approach is now seen as a model example of how to adopt AI effectively within game service operations.

ℹ️ Can We Apply Too?

You don’t need to write complex code to benefit from user feedback analysis. Simply using AI tools to organize data and identify key patterns can directly improve both the speed and quality of your work.

If your company has an AI or data team, this type of project can be developed in collaboration with them.

However, even if you don’t have a dedicated team or don’t need highly advanced features, you can still start small using open tools like GPT-4, Claude, or Perplexity.

Just keep these two principles in mind:

- Write clear, structured prompts to guide the AI effectively

- Always anonymize user data to protect privacy and stay compliant

♻️ Final Thoughts

Generative AI is most effective not when treated as a trend or technology experiment, but when used as a tool to improve the service itself.

At the end of the day, games are a service. And service means understanding players and responding at the right moment.

AI doesn’t replace that responsibility, it helps you do it faster and more accurately.

In the next post, we’ll look at how AI can help with repetitive tasks like competitor analysis, proposal writing, and internal reporting, which game service professionals often face.

※ Disclaimer: This content reflects the author’s personal views and includes only publicly available examples. It does not represent the official position of any company mentioned