Usability Evaluation of FIDO2 on Smartphones

Our paper, “FIDO2 the Rescue? Platform vs. Roaming Authentication on Smartphones,”As a co-first author of this work, I led the overall research design, management, and writing process of the paper together with my colleague Leon Würsching. My major contributions were in the user study design, the quantitative evaluation in R, and the qualitative evaluation of the questionnaire answers. was just pub­lished in the Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems.We are honored to have received a Best Paper Award at CHI’23 for our work. The full paper is freely available here.

This work is in­creas­ing­ly rel­e­vant as all major smartphones now support FIDO2 web authentication, with more and more websites implementing the corresponding WebAuthn W3C standard. With strong authentication options now finally available on smartphones, the question arises: “How usable is FIDO2 authentication on smartphones? Do users accept this new form of authentication, or are there any pain points?” Our lab study provides answers to these questions and shows remaining adoption barriers that prevent FIDO2 from succeeding password authentication.

The following is an excerpt from our paper. The full paper contains much more details.

Introduction

Authentication is one of the central building blocks for securing the Internet. Since the 1970s, web services have conventionally relied on text-based passwords as the de-facto standard for user authentication. Secure passwords, however, are hard to memorize, password reuse enables severe attacks, and password-based authentication is prone to phishing. Despite stronger security guarantees, all previous approaches in the quest to replace passwords failed to get widespread adoption, mainly because of their inferior deployability and usability.

To address the first entry barrier — deployability — more than 250 technology companies and browser vendors have founded the FIDO Alliance, jointly designing the FIDO2 standards for passwordless authentication. As a result, all major web browsers are now FIDO2-ready, as they support the corresponding W3C Web Authetication standard. FIDO2 mandates public-key cryptography to provide user authentication based on authenticators containing the user’s private key. Unlike passwords, FIDO2 authentication is resistant to phishing, keylogging, replay attacks, and server breaches.

As more and more websites support FIDO2 authentication, it becomes essential to study the second entry barrier — usability — to find out how users react to this paradigm shift from knowledge-based factors to possession-based and biometrics-based factors.

Background

There are two variants of FIDO2 authentication with fundamentally different user interactions:

For roaming authentication, the private keys are stored on an external roaming authenticator, e.g., a YubiKey, which connects to the client device via USB, NFC, or BLE. A recent usability study within the desktop environment by Lyastani et al. suggests that users accept and prefer roaming authentication as an alternative to passwords. Many participants, however, were concerned about carrying a security key with them physically and potentially losing it. Despite the prevalence of FIDO2 support in recent iPhones and Android devices, smartphones have not been studied as FIDO2 clients for roaming authentication yet, but only as roaming authenticators themselves.

The second variant of FIDO2, platform authentication, mitigates these availability concerns, as the smartphone’s integrated TPM stores the private keys, protected by an additional local authentication using the smartphone’s unlock mechanism, e.g., Apple Touch ID. However, platform authentication raises new usability concerns about the fundamentally different mental model. Users need to grasp a more complex authentication method combining a private key stored on the smartphone’s TPM that is further protected by biometrics-based local authentication. Platform authentication on smartphones has received little attention so far. Oogami et al. studied how to improve compatible websites’ user experience, and Lassak et al. developed smartphone notifications addressing platform authentication misconceptions. However, the research community lacks an understanding of whether users understand, accept, and trust passwordless platform authentication as an alternative to roaming authentication and passwords.

Contributions

In this work, we try to bridge this knowledge gap by reporting, to the best of our knowledge, the first large-scale lab study comparing FIDO2 platform authentication and roaming authentication on smartphones. We recruited 87 participants, randomly assigned them to one of two groups, and had them perform a series of practical web authentication tasks on an Apple iPhone. The first group used platform authentication with Apple Touch ID for the web, while the second group used roaming authentication with an NFC-based YubiKey. The participants reflected on their experience in a survey, which featured a combination of quantitative questions using standardized metrics and qualitative open-ended text questions. This paper presents our work as follows:

1. As our main contribution, we conduct the first large-scale lab study that compares FIDO2 platform and roaming authentication on smartphones.
2. We show that platform and roaming authentication have excellent usability on smartphones, but lay users generally prefer platform authentication.
3. From our questionnaire’s qualitative categories, we identify the strengths and weaknesses of FIDO2 on smartphones. Based on our participants’ feedback, we investigate how to address the weaknesses of passwordless authentication and discuss account-specific adoption decisions and usage patterns.
4. We provide a replication package with our evaluation scripts and the pseudonymized dataset collected in our study, consisting of 22 variables for each of our 87 participants. We also release our mockup website’s source code to facilitate future work.

Our questionnaire encouraged participants to reflect on their everyday authentication use cases and whether they would be willing to use FIDO2 for their own accounts. While most of our participants were generally willing to adopt passwordless authentication, account-specific adoption barriers remain for both roaming and platform authentication, which we discuss in detail.

Conclusion

“What is the usability and acceptance of FIDO2 passwordless authentication using platform and roaming authentication on smartphones?”

• Both platform and roaming authentication show the potential of satisfactory day-to-day usability on smartphones, but less common events such as device malfunctions, account recovery, and account delegation impair the experience. Overall, users slightly prefer platform authentication.

“What benefits and concerns do users consider when using FIDO2 passwordless authentication on smartphones?”

• Users appreciate platform and roaming authentication as simple and secure password replacements without the overhead of memorizing and managing passwords for each account. The primary weakness identified by users is the loss/theft/destruction of the authenticator and the associated burden of revoking and recovering access to each account. For roaming authentication, users criticize having to carry an additional device. In contrast, for platform authentication, users are concerned with accessing their accounts on other client devices and technical problems with the biometric sensor.

“Which account types do users want to secure using FIDO2 passwordless authentication on smartphones?”

• While most users are likely to adopt passwordless authentication for their accounts, users prioritize usability, security, and availability differently depending on the account type. As a result, the weaknesses of passwordless authentication turn into dealbreakers for specific account types and usage patterns. Participants generally preferred platform over roaming authentication for non-sensitive accounts or frequently used accounts due to the excellent availability and the fast authentication time of the smartphone’s integrated platform authenticator. Conversely, participants preferred roaming authentication for sensitive or rarely used accounts. Some users reject passwordless authentication for shared accounts as they do not see an option to delegate access to other users. Overall, most users prefer passwordless authentication for sensitive accounts. However, despite the usability benefits, some do not fully trust the technology and would therefore only use it for non-sensitive accounts.

In summary, although there is no one-size-fits-all authenticator for all account types, we recommend improving platform authentication, which has more easily addressable weaknesses than roaming authentication.

Read the full paper, which contains much more details.