Announcement

Collapse
No announcement yet.

Unreal Tournament 3 UX test

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    [OFFICIAL] Unreal Tournament 3 UX test

    We had our first UX test for Unreal Tournament, and I wanted to share the results with everyone. Note that this test was specifically about the HUD in UT3, to determine opportunities for improvement with the current UT.

    Project: Unreal Tournament
    Date of the test: 2014/06/26
    Object: Unreal Tournament 3 HUD and overall UI (external test) 

    EXECUTIVE SUMMARY
    6 external hardcore gamers played UT3 for a total of 8 rounds of team death match and one round of capture the flag. The main objective was to test the HUD and UI (signs and feedback) for elements that should/shouldn’t be translated into the new UT. This UX-test was not about gameplay, balancing, or menus.

    MAIN ISSUES FOUND

    Combat/Health:
    o Signs and Feedback on damage and critical health are unclear. Health statistic is lost in number overload on the HUD. Ammunition, health, kill count, team kill count, and enemy team kill count are all conveyed using only numbers.
    o Few players showed any reaction toward low health.
    o Signs and Feedback on crosshair are unclear. Crosshair only turns red as a feedback on hit. There’s no feedback on enemy range or shooting.
    o The crosshair should better convey how the equipped weapon is working.

    HUD/UI:
    o There’s a red overload, especially for the red team since the whole HUD is red.
    o Text is not always legible, there’s no text box and font is difficult to read.

    Controls:
    o Overall the controls seemed easy to learn. However, there is a lack of consistency in how some weapons work (LMB vs. RMB vs. RMB hold…), especially the charging functionality (when there is one), and most players didn’t discover it.

    - Gameflow:
    o Players were seen attacking their own teammates. This is especially problematic for players who were on the blue team then switched to red team. Red draws attention and is interpreted as danger. It takes more workload for the brain to inhibit that first reaction.
    Jim Brown
    @EntropicDev
    Epic Games

    #2
    As this thread is in the "discussion" section, are we about to discuss any UX of the HUD?

    What I see by looking at the results, are these hardcore gamers coming from a Shooter with ballastic weapons?
    ] Map Scaler Tool | Betrayal for UT4 | No Spawn Protection | No Pickup Timer | BioLauncher (revived) | ForcePickupSpawn | Map cosmetics::P | Safe Spawn::P | Why numbers for Health/Armor suck!::ANALYSIS/CONCEPT
    ] UT3 Addons: NoMoreDemoGuy | PickupRespawnTweak | Mutate Spec | MutePawnSounds | NoPlayerBeacon | Epic FTW | Epic FOCK | TripodSound (... and many more)

    Comment


      #3
      I take issue with this:

      Originally posted by Entropy View Post
      o Signs and Feedback on crosshair are unclear. Crosshair only turns red as a feedback on hit. There’s no feedback on enemy range or shooting.
      o The crosshair should better convey how the equipped weapon is working.
      Why should we rely on the crosshair to give user feedback? The crosshair serves a critical function that trumps all other considerations: aiming.

      Feedback on how a weapon works and what you are hitting is hurting more than helping if it decreases weapon accuracy. That is why almost all competitive players use some form of outlined dot or similarly simple crosshair.

      I'd also like to point out that while red certainly has cultural implications, the eye is most sensitive to green. For UT comp brightskins I always set them to bright green, because that makes them stand out the most. Red on the other hand is one of the least sensitive colors for your eyes. So when everything is red it is hard to decipher individual elements. Magenta is another interesting color because it is a construct of your brain, and immediately stands out.
      Last edited by TheWhiteDragon; 07-03-2014, 11:57 AM.
      BeyondUnreal Podcast
      r/UnrealTournament Moderator

      Comment


        #4
        Originally posted by RattleSN4K3 View Post
        As this thread is in the "discussion" section, are we about to discuss any UX of the HUD?
        Yes, please do!

        Originally posted by RattleSN4K3 View Post
        are these hardcore gamers coming from a Shooter with ballastic weapons?
        Here's the participant info (names removed to protect the innocent):


        Attached Files
        Jim Brown
        @EntropicDev
        Epic Games

        Comment


          #5
          If these guys are hardcore i'd hate to see what your idea of a casual gamer is

          I mean come on, they got confused as to what team they were on and shot each other

          Are you looking at how easily people with no idea what they're doing, but have played other FPS games made recently cope with the UT3 hud?

          Here's something your study won't tell you - you play better when you're relaxed, looking at relaxing colours like blue helps you play well in the same way that looking at red makes you more aggressive and altert to danger

          It's a subtle trade off which after 15 years and god knows how long in other games hasn't produced a single issue

          If it's a problem, put other colour options in the hud, i use grey and i don't notice that the hud is there until i want to notice it

          Critical health works best in the UT1 hud(hurray for UT1) it flashes to indicate that your health is low, you can miss this sometimes if the situation is intense but even your 'hardcore' guys should see something like this

          Next time try getting them to play A RANGE of UT games with different HUD setups each time, then get feedback

          Comment


            #6
            Originally posted by Entropy View Post
            We had our first UX test for Unreal Tournament, and I wanted to share the results with everyone. Note that this test was specifically about the HUD in UT3, to determine opportunities for improvement with the current UT.

            Project: Unreal Tournament
            Date of the test: 2014/06/26
            Object: Unreal Tournament 3 HUD and overall UI (external test) 

            EXECUTIVE SUMMARY
            6 external hardcore gamers played UT3 for a total of 8 rounds of team death match and one round of capture the flag. The main objective was to test the HUD and UI (signs and feedback) for elements that should/shouldn’t be translated into the new UT. This UX-test was not about gameplay, balancing, or menus.

            MAIN ISSUES FOUND

            Combat/Health:
            o Signs and Feedback on damage and critical health are unclear. Health statistic is lost in number overload on the HUD. Ammunition, health, kill count, team kill count, and enemy team kill count are all conveyed using only numbers.
            o Few players showed any reaction toward low health.
            o Signs and Feedback on crosshair are unclear. Crosshair only turns red as a feedback on hit. There’s no feedback on enemy range or shooting.
            o The crosshair should better convey how the equipped weapon is working.

            HUD/UI:
            o There’s a red overload, especially for the red team since the whole HUD is red.
            o Text is not always legible, there’s no text box and font is difficult to read.

            Controls:
            o Overall the controls seemed easy to learn. However, there is a lack of consistency in how some weapons work (LMB vs. RMB vs. RMB hold…), especially the charging functionality (when there is one), and most players didn’t discover it.

            - Gameflow:
            o Players were seen attacking their own teammates. This is especially problematic for players who were on the blue team then switched to red team. Red draws attention and is interpreted as danger. It takes more workload for the brain to inhibit that first reaction.

            Well it's clear something has to be done with the crosshair. I would like the crosshair to be always the same, in UT99 I use a red dot, that makes shock rifle combos easier than with other crosshairs.

            And btw, just look at the first minute of this video and tell me if you see the different crosshairs:

            https://www.youtube.com/watch?v=xVOPs8n5vwo
            Last edited by velocidad; 07-04-2014, 03:19 AM.

            Comment


              #7
              Relative Team Colors

              Enemies and Teammates should have preset team colors that make sense (still configurable).
              Spectators always see absolute team colors (ex: red and blue). There's no issue with red conflicts here.

              Red is probably the default dominant color used to convey threat or taking damage, so it won't change, and should therefore be avoided as much as possible in other contexts.

              Characters and text/symbols should have separate color controls.
              * Maybe you want gray HUD symbols/text messages for enemies, but keep green for characters.

              Hit Colors

              How does the game communicate that the enemy has shield/armor?
              * Does it flash a fixed color?
              ** That would break relative color choices.
              * Does it spawn some shield effect? What color to use?

              Team Color Reminder

              What team are you on?
              * Already solved by the fixed relative teammate/enemy skin colors?

              Other options:
              * Have the display fade in starting from your team color when you spawn.
              * A glow can be shown at the top part of the HUD when spawning, like when you chose the dark side/light side in SWTOR, though not as intense.
              * Your team info is always on the left side (so you know that this color always represents your team), and enemies on the right; in other words, your team info is always at the same location.

              HUD Tint

              * Widgets should probably share a neutral color (dimmed gray) by default, then increase in brightness based on what elements update more frequently/are more important. The weapon bar should be dimmed, as it's a constant part, and rarely conveys any critical or important information. Your held weapon has its own ammo/resource count widget which is brighter. etc
              * Experiment with optimizing brightness toward what elements are updated most frequently.
              Ex: Team scores in a round-based team game are not updated often, thus would not need emphasis.

              Localization

              Although spectators would always see red and blue by default, maybe go over to using color neutral designations:
              Alpha Team / "Alpha Team Scores" / ...
              Bravo Team
              Charlie Team
              Delta Team

              * Avoids "red" and "blue" colors (even though spectators have to see some preset colors).
              * Still no need to add relative announcements such as "your team" and "enemy team" in addition to "red team" and "blue team" etc.

              Low Health Warning

              I'm not sure if this is useful in UT or needs much emphasis, because the action is too quick for it to be something to act upon at times.

              Characters typically die fairly quickly in close combat--too quickly for such a warning to have any helpful effect in the way it's been implemented so far.
              Vehicles typically have more armor/health, makes more sense here, perhaps.

              It would depend on the algorithm that detected low health.
              Maybe the algorithm could use the rate at which you lose health as a warning trigger, instead of using a fixed health threshold alone.

              Comment


                #8
                @Entropy
                Thanks for the additional info about the participants.

                It's a good list of different games but none of these is like UT - a unrealistic Jump'n' Run shooter with cheating-like weapons .




                IMO, the only feedback a crosshair should convey is the center of three sceen and what to expect when you shoot that weapon. The enemy range or shooting is irrevelant for UT as no weapon has much effect on spread or recoil, the distance is irrelevant for instant hit weapons and most of the others.

                Note: This is q quick UX with any form just by looking at the menu as being a new player. Most of the things are not my personal preference, but that's not what UX is all about. A UI should not be that different to the real world or to other common elements in other games.

                Only a small portion of the weapons need crosshair tweaks.
                The rocketlauncher is one nice example which has a proper feedback. Only the tertiary firemode indication is missing (RMB hold + LMB). A nice animation when the weapon is loading ammunition, and a quick text field stating which mode is currently set.
                Crosshair
                • Charged weapons should have an animated (e.g. pulsing scaled crosshair)
                  for an addication feedback even if the weapon is hidden
                • Range weapons should inidcate when the range hit its limit (e.g. LinkGun alt)
                  most of new players do attack with Alt-Fire even when the target is out of range
                • Crosshair should not be scaled when items are picked up
                  it's used in other games for spread


                Weapon bar
                The weapon bar is good in its state but there are some minor things.
                • - Groupd weapons are not shown (e.g. Avril and Deemer uses the same slot)
                • - Low weapons are not indiacted (e.g. yellow/red ammo bar)
                • - If deployable is taking, the not-allowed-weapon-switch is not shown
                  if a pickup is held, you can switch weapon, there is no helping message (like for entering a vehicle while carrying the flag)
                • - Missing indication for "best weapon"
                  might help for starters to know what they can expect for different actions (switch to best weapon, switch the last, prev/next weapon)
                • - Ammo bar does not show an icon
                • - Ammo only shows only absolute ammo number (e.g. combo uses 5 rounds)
                  if compared with a grenade launcher which has split ammo, the available combos could be listed as well)


                Weapons
                • - Avril doesn't show how to use the Lock-on
                • - Lock-On for rocket launcher is secret
                • - Impact Hammer's EMP has a bad feedback which is quite unknown unless you read the manual (or code)
                • - Spread for Enforcer's continueing fire should be drawn (shooting while holding LMB has a spread, only a intervalled shot is reliable to hit the center of the screen)
                • - Bio Rifle's crosshair could be animated to show charged ammo
                • - LinkGun's alt fire healing is not drawn ("Fire with 2nd fire mode to heal vehicle")


                Misc
                • - Post Processing should not effect the HUD. The deemer explosion adds a shake to the player's view which offests the crosshair for about 2 seconds.
                • + Damage indication is subtle and easy to understand
                • + All UT3's crosshair icons do make sense if you take a closer look and played the game for some time.
                • + If target is hit, the crosshair turns red, which isn't distracting but helping alot
                • + Good semi-transparent hud for increased the visible area of the actual game
                • + Instant respawn effect
                  ever played Battlefield? You die before you screens fades in there.
                • + You actually notice when you died (because of the 3rd person camera switch)
                • - No indication when you are able to be heared by others (as footsteps are not played locally)
                • - Missing indication for the ability to double jump (like the Scorpion does when he's ready to boost)
                  only a small portion might notice the capability to do a double jump due to mass key pressing in a danger situation
                • + Colored HUD for team games
                  You must be blind if you don't notice on which team you are playing if everythin is colored in your view
                • - Missing separate chat log (which is a problem in most of the multiplatform retail games) which get filled quite fast with the default settings of having death-message drawn
                  It's only available in menu or console, but the console is filled with every message.
                • + No view shake for real weapons (only Flak-direct-kill and Hammer had it)
                  If you shoot a weapon in UT its not doing anything to the actually view different to most of the games listed in the table above (from the UX test participants). The view should give exactly the right feedback of what is happening -> no recoil/spread/etc. = no view shake
                • - Missing ingame helping message for pickups (for starters)
                  The ammo pickups are placed in the map but are not recognizable as pickups (like Enforcer ammo in DM-Deck) ... only if the glow effect starts you see the ammo, but that's the same glow effect like most of the environment objects. From a visual design aspect, it looks good and fits well but the important part is the gameplay.
                  Something like "Pickup Enforcer ammunition for another 15 rounds" to be drawn if you aim the pickup and standing nearby.
                • - Pickup messages can be missunderstood
                  Once you move onto a pickup, it justs hides. There is no animation for "picking up" that item like it would be in Role-playing games. If a new player just freelance around the map they get a message but never know directly what it was as these message are at the same place like weapon switch, game messages, etc.
                • - Missing visual feedback from which enemy you got killed
                  Hint: Talking about Death cam which helps new players a lot (Titanfall or Call of Duty). Or something like what Blacklight: Retribution did - on death slowing down the local game and add a transition from the death location to the killer... if allowed by the server.
                • - Missing critical damage indication (if hit by sniper or combo and HP is low).
                • - Actual score point for a kill/capture is not drawn
                  like the "point list" in TitanFall, CoD, ...


                I think a complete tutorial should be added where there are challanges for players in which they can also compete online against other like Time-trial, fire accuary in Combos etc. TitanFall has a great (but minimal) forced tutorial in order to play online.
                UT2004 had nice tutorials (but mostly for gameplay). I had ideas for my UDK port of UT3 adding several interactive tutorials which I would have added as "Bootcamp" which would feature:
                • Tutorials: a full-blown in-game tutorial with helping messages for general gameplay, map based gameplay, some tips and tricks for starts etc, also
                • Training: something like UT3 campaign but without the story but several helping message. This would have been a separate game type which annotates several ingame objects like players, routes etc.
                • Challenges: map-based or general challenges in order to gain experiance and increased skill. Like "Do a hammer jump form here to there", "Infiltrate base without being shot", "Kill 3 enimes with an air combo"


                One thing to mention. A user-experience test made with hardcore gamers but at the same time asking for needed information for casual gamers isn't that reliable. The hud should work for everyone even for those who never played any game before. If you only ask hardcore games, the competitive scene, you only get a product of the user-friendlyness of like Android, but we all know you need something like iOS ;-) (even iOS had lacks for instance the pinch-out-in input or press-and-hold).
                ] Map Scaler Tool | Betrayal for UT4 | No Spawn Protection | No Pickup Timer | BioLauncher (revived) | ForcePickupSpawn | Map cosmetics::P | Safe Spawn::P | Why numbers for Health/Armor suck!::ANALYSIS/CONCEPT
                ] UT3 Addons: NoMoreDemoGuy | PickupRespawnTweak | Mutate Spec | MutePawnSounds | NoPlayerBeacon | Epic FTW | Epic FOCK | TripodSound (... and many more)

                Comment


                  #9
                  Interesting results. Thanks for sharing …



                  Originally posted by Entropy View Post
                  MAIN ISSUES FOUND

                  Combat/Health:
                  o Signs and Feedback on damage and critical health are unclear. Health statistic is lost in number overload on the HUD. Ammunition, health, kill count, team kill count, and enemy team kill count are all conveyed using only numbers.
                  o Few players showed any reaction toward low health.
                  This is a legitimate concern; new and even middling experienced players often show little awareness of their health status.

                  Wildly flashing/jittering health indicator is the go-to fix, which is fine, although IMO screen overlays (vignetted bloodied retina effect, etc.) are the most effective solution, short of physically debilitating the player's movement. The user is properly annoyed and inconvenienced by the overlay effect, to where it cannot go unnoticed, and is more likely to induce a change in the user's behavior.


                  Combat/Health:
                  o Signs and Feedback on crosshair are unclear. Crosshair only turns red as a feedback on hit. There’s no feedback on enemy range or shooting.
                  o The crosshair should better convey how the equipped weapon is working.
                  Draw crosshairs in three passes, each of which may be optionally disabled by the user:

                  (1) decorative feedback, i.e. health/armor charts, etc.;
                  (2) the familiar UT semi-transparent, unique-per-weapon animated crosshair; and
                  (3) an opaque, high-contrast pinpoint dot.

                  Displaying the name of the targeted opponent while in range is sufficient feedback.


                  HUD/UI:
                  o There’s a red overload, especially for the red team since the whole HUD is red.

                  - Gameflow:

                  o Players were seen attacking their own teammates. This is especially problematic for players who were on the blue team then switched to red team. Red draws attention and is interpreted as danger. It takes more workload for the brain to inhibit that first reaction.
                  Decouple team color from team identity.

                  Make [red|blue … or green|magenta, etc.] a fully client-side rendering artifact. Refer to teams in-game (localized messages, announcers, etc.) by their identity, not their color (e.g. "Thundercrash Flag returned", "Axon Power Core is under attack", etc.).

                  As a matter of level design discipline, discontinue use of static colored assets (red|blue banners, wall textures, etc.), in favor of team-logos and dynamically colored materials.


                  Controls:
                  o Overall the controls seemed easy to learn. However, there is a lack of consistency in how some weapons work (LMB vs. RMB vs. RMB hold…), especially the charging functionality (when there is one), and most players didn’t discover it.
                  This is solved with tutorials.

                  The weapons in UT are diverse on purpose, and externals are simply not accustomed to this. It is too much to dumb-down the controls for the sake of shoehorning in some shallow "consistency".
                  Last edited by Veggie_D; 07-03-2014, 02:54 PM.

                  Comment


                    #10
                    Love UX threads!

                    I wish someone made a UT3 mutator to fix all the things mentioned here (especially the crosshair stuff), that'd be awesome.
                    Snir "DDRRE" Hassidim - Living and breathing UT since 2002!
                    "When Video Games and the Enterprise Meet" - Read my chronicles of building a gamers club at my workplace.
                    Facebook | LinkedIn

                    Comment


                      #11
                      o Signs and Feedback on damage and critical health are unclear. Health statistic is lost in number overload on the HUD. Ammunition, health, kill count, team kill count, and enemy team kill count are all conveyed using only numbers.
                      Health can be indicated through a black and white screen if you reach low health (e. g. 20%). That way you can get health feedback AND you reduce the red overload.
                      Ammunition can be displayed through an animated bar which is full if you have max ammo capacity.
                      Team kill count and enemy team count can be displayed with a bar too showing the colors of each team. If both have the same number of kills the bar is 50% red and 50% blue. In the colored bar you can still display the numbers.

                      o Signs and Feedback on crosshair are unclear. Crosshair only turns red as a feedback on hit. There’s no feedback on enemy range or shooting.
                      Maybe use turning red for range indication.
                      For hits you can use the "Ping" sound and as visual indicator an addition to the crosshair change (e. g. if you have a crosshair like () it will change for a short sec to <()> if you hit the enemy)

                      o There’s a red overload, especially for the red team since the whole HUD is red.
                      o Players were seen attacking their own teammates. This is especially problematic for players who were on the blue team then switched to red team. Red draws attention and is interpreted as danger. It takes more workload for the brain to inhibit that first reaction.
                      I would go with the suggesstion of always be on the blue team and enemy is always displayed Red (or Orange).
                      Especially if you play several rounds on Blue team and than switch the team you will always shoot accidentaly sometimes on team mates. I think this will not even change with further team indicators such as the HUD being in team colors.

                      o Overall the controls seemed easy to learn. However, there is a lack of consistency in how some weapons work (LMB vs. RMB vs. RMB hold…), especially the charging functionality (when there is one), and most players didn’t discover it.
                      Introduce an option where tutorial messages are displayed on screen. When you switch to an other weapon a short functional introduction is given near the weapon on the screen. This can be deactivated by advanced players in the options.
                      An other option would be to display some tips & hints during loading screens. There you could show the hint that some weapons have a charging functionality.

                      Comment


                        #12
                        Regarding players attacking their own teammates, perhaps this is due to these players simply not playing a Red vs. Blue game in such a long time? It obviously worked for previous UT games, and even Halo used the Red vs. Blue scheme.

                        Seems like players may just need a small amount of time to readjust some of these things again.

                        Comment


                          #13
                          As for red overaload in GUI for red team. Team based coloring of GUI (which may be reminding team alignment) doesn't have to colorize whole interface. Coloring can be still mostly neutral, but some minor elements of can be colorized.
                          Unofficial Enforcer Model
                          Redesigning Enforcer Firemodes

                          Comment


                            #14
                            Honestly, I think you are over-thinking this whole "red overload" issue. Before games like CoD and BF came around, many of the MP games that had team-oriented game modes have used the red vs blue paradigm almost exclusively. All the Quakes, all the UTs, Team Fortress, Halo, etc. So why is this an issue now? This whole idea that players are going to start attacking their own teammates when they switch teams feels extremely condescending, especially towards new players. You're underestimating their ability to adapt to new situations. Sure it might happen once or twice but they will quickly adapt to it, just like the rest of the people who played these games years before.
                            http://i.imgur.com/cv5BV2r.png

                            Comment


                              #15
                              I think the perceived "overload" may be a result of the huge red hit flash effect. Then again, I wouldn't mind having orange/blue as the primary team colors, or maybe even purple/green. Or something entirely different.

                              I think the most important aspect about this entire UX thing: This was a "first reactions" test. You should repeat that with three control groups of players who primarily played UT, UT2004 and UT3 respectively. There is no other way to figure out how well players may adapt to the concepts over time.
                              <elmuerte> you shouldn't do all-nighters, it's a waste of time and effort
                              <TNSe> nono
                              <TNSe> its always funny to find code a week later you dont even remember writing
                              <Pfhoenix> what's worse is when you have a Star Wars moment
                              <Pfhoenix> "Luke! I am your code!" "No! Impossible! It can't be!"
                              <@Mych|Lockdown> ...and the award for the most creative spelling of "Jailbreak" goes to ... "Gandis Jealbrake Server"

                              Comment

                              Working...
                              X