The Hackathon Method

September 22nd, 2023

Introduction

Why do research

The PGP Framework

How to define a persona

How to define a persona’s goals

How to define a persona’s problems

The Hackathon Method

How to recruit participants

How to conduct an interview

How to reflect on what you learned

How to present to hackathon judges

Introduction

Is the product you’re building valuable? Is it usable? Is it effective? Are you unsure how to go about answering these questions?

In this guide you’ll learn efficient ways to think like a UX researcher and de-risk your product idea. We wrote this article for hackathon participants, but anyone building a new product should be able to take advantage of these steps.

Let’s get started.

Why do research

Most solution ideas aren’t compelling to their intended users, and of the few that are, even fewer are shipped in a package that can scale.

The surest way to position your product idea for success is to get unbiased user feedback early and often.

And before you collect feedback, it’s wise to document how you think your solution solves a problem. Writing your assumptions down upfront will help you maximize how actionable your feedback sessions are.

The frameworks below include simple structures to document your assumptions, get immediate feedback, and build confidence in how well your idea fits the market.

The PGP Framework (Persona, Goal, Problem)

A simple way to understand a product’s impact is to define its target users, their goals, and the friction they run into when trying to achieve those goals.

Example:

Persona: A person getting paid in DAI who also has a debit card that spends ETH
Goal: Minimize fees when spending crypto on debit card
Problem: Transferring DAI to ETH is costly and time-consuming

An important step to any product research is to define these parameters, so you can identify the right kind people to give you feedback.

How to define a persona

To define a persona, ask yourself, what externally observable behaviors do expect users of your product will have? As much as you can, try to think of behaviors that meaningfully separate users you want to focus on vs. users you don’t.

Example:

Onramps with MetaMask Buy
Mints at least 1 NFT a month
Uses Opensea as their primary marketplace

Do this now: Create a list of 5-10 behaviors that potential users exhibit. You’ll use these behaviors to determine who to talk to later. Feel free to give this persona a name, like “Defi Degen“, “NFT Artist“, etc…

How to define a persona’s goals

To define a persona’s goals, ask yourself, when your target user is in a particular context, which goal of theirs are you supporting?

Example:

When NFT artists are listing digital art on a marketplace, they want to notify past collectors that new work is available.
When DeFi degens are talking to their accountants during tax season, they want to share transaction history with accountants.

Do this now: Think about what types of goals your persona likely has. Don’t just pick any goal, think about the situations/contexts those users might find themselves in and think about what types of goals they have in those situations. Create a list of 1-5 goals.

How to define a persona’s problems

To define a persona’s problems, ask yourself, as your target user pursues the goal you’re focused on, what frictions or problems are they running into?

Example:

When NFT artists want to notify past collectors that new work is available, they have to manually scan smart contract transactions to find all accounts who minted or bid on their work
When DeFi degens want to share transaction history with accountants, they have to manually add currency outflows from multiple wallets

Do this now: Put yourself in the shoes of someone with the Goal in question. What types of behaviors, activities, or mindsets do they exhibit in pursuit of this goal, and what interrupts them? Which of these interruptions feel particularly meaningful to you? Create a list of 1–5 problems that you believe are most significant for your persona achieving their focused goal.

How PGP fits into your next steps

Once you define the PGP elements you’re building for, you can reference them when you look for people to talk to. Only invite people to talk to who share the goal you’re focused on. And if you want to get really specific, filter out people who don’t experience any of the problems.

The Hackathon Method (Recruit, Interview, Reflect)

The Hackathon method is an efficient way to get the most relevant people to give you actionable feedback in a short, in-person session.

At a high level you will:

Recruit (Find users to test with)
Interview (Test with users)
Reflect (Examine what you learned and consider changes)

What follows is everything you need to execute this method with a high degree of confidence.

How to recruit participants

Hackathons are crowded events which is ideal for quick user feedback sessions. After following this guide, you should feel confident finding 5 people to talk to.

How to approach people

Before you start inviting people to give you feedback, there are a few things to keep in mind:

Most people aren’t there to test your product
Most people might not be a good fit
Most people might not have time to test with you

Asking for someone’s time is always hard. People come to these conferences for different reasons. Before you ask for 15 minutes of their time, take a few minutes to chat with them with them about conference and build rapport.

Example:

Say hello: “Hey! My name is X, what’s your name?” [Response]. “Are you enjoying the conference so far?” [Response]. “That’s amazing to hear! We’re working on a submission for the hackathon, I was wondering if I could take 2 mins of your time to ask you few questions?”
Qualify: If they say yes, you can start asking them your qualifying questions.
- See the next section of this article for help writing qualifying questions
Ask for 15 minutes: If they’re a good fit, you’re going to need a larger chunk of time to ask more pressing interview questions so you can get proper feedback. If they’re not a good fit, tell them thanks and look for someone else to speak with!
- See the next section of this article for helping writing interview questions
Reward: Our research shows that participants are much happier and will be more patient with you if they know they’re getting a reward for their time. Offer to buy them a coffee, a snack, or a free NFT. You can say something like, "Hey, I think you fit our user profile really well. Do you have 15-20 mins to help test our submission? I'd love to buy you a coffee or snack for your time. You'd really be helping us out."

How to qualify people

Once you’ve approached someone and they’ve agreed to spend 2 minutes with you, it’s time to ask them 3–5 qualifying questions to make sure they’re a good fit.

For example: Say your persona definition is MetaMask Users that have used the “buy” feature at least once before.

You might ask the following questions to determine if someone fits the definition or not.

What wallet do you use?
How long have you been using that wallet for?
How do you get crypto into that wallet?
Have you tried other ways to do this?

Note that we did not mention “MetaMask” or the “Buy” feature in any of the questions above. It’s always best to avoid giving away details that might bias your participant into accidentally giving you a wrong answer.

After asking your questions, it’ll be obvious whether the person you’re speaking with is a good fit. They may not even fit your definition fully, and that’s okay. As long as you’re taking time to qualify them in some way, you can be a bit flexible with participants.

Do this now: Put it all together. First, prepare your qualifying questions. Then, go talk to 5 people and ask them if they can spare 2 minutes of time. Ask your qualifying questions, and if they are a good fit, offer them a reward and your gratitude for spending 15 minutes with you.

How to conduct an interview

Understanding goals, behavior, and problem severity

One of the most important parts of a user feedback session is getting people to talk about their problems in context.

This isn’t therapy. You don’t need to learn about every problem that our participants have. The purpose of this section is to get unbiased answers on the following topics:

How meaningful is the problem you’re focused for your audience?
How does your audience solve the problem you’re focused on today, if at all?

You can’t just ask people these questions outright—you won’t get reliable answers. You need people to describe their personal goals, behaviors, and context without bias, THEN, you can infer how their answers impact your core questions.

How to ask about goals

Ask them about the Goals they have in your solutions general domain. Do they mention the ones you’re focused on without you mentioning it to them?

Examples:

What do you know about [solutions general domain]?

“What do you know about onramping?“

When was the last time you [had X goal in solution general domain]?

“When was the last time you had to convert FIAT to crypto?“

How to ask about behavior

Ask someone about the last time they did the Behavior you’re talking about. Do they mention it?

Example: Tell me about the last time you [did behavior + any relevant context]

“Tell me about the last time you on-ramped into crypto.“
“Walk me through the last time you minted an NFT. Where were you? Which tools did you use?“

How to ask about challenges

How painful is the problem for them? Ask them to rank it 1–5.

Example: Describe a challenge you faced while [doing some behavior to achieve a goal].

“Describe a challenge you faced onramping into crypto.“
“How much of a challenge was that for you on a scale from 1-5?“
“Can you describe another challenge?”

As you ask participants questions about their goals, behaviors, and problems, there are a few important behaviors you should follow:

Use open-ended questions (Rule of thumb: Avoid asking questions that are easy to reply “Yes” or “No” to)
Avoid leading participants to answer in a certain way (The answer should not be implied in your question)
Remind participants there are no right or wrong answers.
Ask for examples from the past.
Listen more than you talk.

Do this now: Put this all together to write a few questions exploring the goals, behaviors, and severity of problems your participants face. It might look something like this:

“When was the last time you had to convert FIAT to crypto?“

“Walk me through that experience. What prompted you to do it? Where were you? Which tools did you use?“

“Can you describe a challenge you faced while onramping?“

“How much of a challenge was that for you on a scale from 1-5?“

“Can you describe another challenge?”

Understanding solution “fit” and usability

Another important opportunity during a user feedback session is to get participants to actually use your product. This is your chance to test both the concept of your solution (how well it “fits” the problem you’re focused on), and how easy it is to use.

Provide a narrative to contextualize your tasks

You can’t just drop participants into your submission and ask them to figure it out. You have to create a realistic narrative for them to follow, and a specific task they should complete inside your solution. Here’s an example:

“Imagine you’re at the grocery store and you’re about to pay for your items. You realize the store accepts crypto payments and you recently installed a new app that lets you pay for grocery items with crypto from your phone. Open up the app and attempt to pay for your groceries. Take your time and speak out loud as you think about look around the interface. There are no wrong answers and we want your honest and direct feedback.”

As you give users tasks to complete, there are a few important behaviors you should follow:

Remind them to think out loud as they navigate your solution. This is not a common behavior for most, so if they stop, continuously probe them to describe out loud what they think is happening, what will happen next, and how they’re feeling overall.
Let your participant struggle in key places. You’re not trying to trouble-shoot your participants to success. Instead, you’re observing how they make sense of and ultimately use your solution. You will need to take the training wheels off to get the best feedback. Sometimes it’s best to shut your mouth and watch
Remind your participant that you want honest feedback. Because you’re allowing users to struggle, it’s important to remind participants that they’re not being graded on performance. This is not a speedrun. Your goal is to have them complete the experience as naturally as possible.

Take notes on key moments during their experience

As you observe participants work through the tasks you provided from the section above, it’s wise to take notes on their thoughts and behaviors in a structured way. What you take notes on might change based on what you want to learn most, but here are some examples of things to write down for an average study:

Do this now: Pick 2-3 of the biggest tasks and write them down. You’ll be giving these to your users later. For example: If you’ve built a wallet, you’ll likely want users to test these tasks

Initial Wallet Creation

Sending Transactions

Connecting to dApps

Starting your interviews

If you’ve read the sections above, you should be ready to start interviewing. Follow the guide below to kick things off.

Do this now: Here’s a quick checklist start your first interviews:

Define your Persona, Goal, and Problem (PGP Framework)

Prepare your Qualifying Questions (Recruit)

Prepare how you’ll approach new participants (Say hello, qualify, ask for 15 minutes, and offer reward)

Prepare your questions on Goals, Behaviors, and Problem Severity

Prepare the Tasks you’ll ask users to complete

Confirm your solution is ready to be tested

Prepare a notetaker to take notes on key moments during the usability test

Go out and find your first participants

Remember to thank your participants and reward them for their time

When you’re done with a few interviews, read the final section of this article to help you gather your insights and package them together into an actionable report.

How to reflect on what you learned

After a few interviews, compile your notes and look for themes spanning multiple participants. Discuss these with your teammates—people with different experiences may infer different insights from the same data.

Valuable

How many users experience the problem you’re focused on?
How severe do your users describe the problem you’re focused on?
How well does your solution improve the problem you’re focused on?

Usable

How successful are users at onboarding to your solution?
How successful are users at navigating the core valuable experience of your solution?

Delightful

What moments of your solution experience spark joy for your users?
What aspects of your solution are so good they’ll motivate users to tell others about your solution?

How to present to hackathon judges

Most hackathon judges want to see you’ve taken some time to consider the value and usability of your solution. Once you’ve conducted interviews and reflected on your learnings, create a presentation slide that shares some of your conclusions.

As you create your slide, consider adding evidence from your user research that clarifies your points on value, usability, and delight. You may consider adding:

Insightful quotes
- Valuable: “I really use MetaMask Buy monthly and always struggle with it.“
- Delightful: “I loved seeing transaction metadata displayed this way. I immediately understood where my coins were going.”
Photos
- If you can get consent, see if you can take a picture of your testing setup with your participants
Screenshots
- Include sections of your solutions where users ran into the most problems. Then show how you made changes that addressed those issues. A before and after will work well here.

Good luck!

Follow OpenUX for more content like this

OpenUX is an agency and community that envisions a world where web3 understands its users.

Join our community to meet likeminded builders, researchers, and strategists contributing expertise and promoting an ecosystem of open-source user research.

Hire us to conduct user research or trainings to help your team understand your users, improve your design, and achieve adoption.

Have more questions? Feel free to write to us at hello@openux.xyz.