Spruce

Introduction

The current fees and escrow process in Spruce’s internal software, Pinecone, was inefficient, requiring too many manual steps, leading to double work and human errors.

These mistakes had a direct impact on Spruce’s bottom line, as any errors in managing transactions could result in financial losses for the company, which operates as an insurance provider.

Problem Statement

Research Approach

We conducted contextual inquiries and usability testing to identify inefficiencies in the fee and escrow processes. These methods provided valuable insights into user workflows and pain points.

Our deliverables included a detailed task analysis, user personas, and recommendations based on usability test results, all of which guided the redesign.

01

02

Contextual Inquiries

I conducted 15 one-hour remote contextual inquiries with Closing, Pre-Funding Associates, and Funding Associates to understand how real estate teams use Pinecone, Spruce’s internal software. These sessions, held via Google Meet, focused on uncovering user needs, pain points, and workflow inefficiencies, with insights transcribed and coded for analysis.

Process

Key Challenges

This high-level user flow shows how the Closing Associate, Pre-Funding Associate, and Funding Associate collaborate to create, set up, and disburse fees. It highlights inefficiencies in the process, where manual steps and repeated document searches across multiple pages cause delays in completing real estate transactions.

Deliverables

One of the main deliverables from the contextual inquiries was the development of a user persona, which helped to encapsulate the core needs, pain points, and goals of real estate professionals. Additionally, we compiled a detailed spreadsheet of usability issues uncovered during the research, which provided clear direction for the design improvements.

This new persona combines insights from three key roles—Closing Associate, Pre-Funding Associate, and Funding Associate—to address shared goals, pain points, and workflow challenges in the fee disbursement process.

User Persona

Usability Issues

This table documents the key usability issues identified during the contextual inquiries, highlighting specific pain points within the fee disbursement process, such as manual data entry, limited functionality, and the absence of autosave features.

These insights helped prioritize areas for improvement to enhance the user experience in Pinecone. Here is the link to the full table of usability issues.

Task Analysis

As part of the contextual inquiries deliverables, we conducted a task analysis of the workflows for the Closing Associate, Pre-Funding Associate, and Funding Associate. This analysis was essential for understanding how to consolidate the fees and escrow pages into a single, combined page that streamlined the process while maintaining all necessary functionality.

Audit Risks for Spruce’s Paper Trail

Error-Prone Fees and Escrow Process

Inefficient Fee Creation & Disbursement

In the event of an audit, Spruce needs transparency to prove how fees were calculated. The current system lacks this, potentially leading to costly errors.

The current process could result in financial loss as Spruce may be held liable when mistakes occur during fee creation and disbursement.

Creating fees, setting them for disbursement, and sending them to various parties involves too many steps, hindering operational efficiency.

03

First Iteration: Building on Contextual Insights

Creating a Fee Workflow

Step 1: Homepage

Step 2: Select Fee Category

Step 3: Attach a Supporting Document

Step 4: Input Fee Information

Step 5: Add Payee Information

Step 6: Verify Fee Information

After conducting the contextual inquiries, I handed off the deliverables to the Product Designer to create Version 1. In this version, users can now create and send fees from a single page, significantly reducing the amount of work. Additionally, users are able to attach supporting documentation to explain why and how a fee is created, serving as crucial evidence in the event of an audit—a feature that was previously missing.

Disbursing a Single Fee Workflow

Step 1: Select Fee & Open Kebab Menu

Step 2: Confirm and Click "Disburse"

Step 3: System Confirms Fee Disbursement

Disbursing a Multiple Fees Workflow

Step 1: Select Fees to be Disbursed

Step 2: Confirm and Click "Disburse All"

Step 3: System Confirms Fee Disbursement

04

Usability Testing

Process

After delivering the discovery research, the first version of the redesigned fees page was created and presented to stakeholders. I recommended a four-week moderated usability test to ensure we were building the right solution.

Each week, 8-10 participants, including Closing, Pre-Funding, and Funding Associates, were guided through three main use cases: fee creation, fee editing, and fee disbursement. These critical tasks were tested to gather feedback and refine the design, ensuring a more efficient and user-friendly process.

Metrics Explained

As part of the usability test, we collected quantitative data to complement the qualitative feedback, allowing us to compare and contrast the effectiveness of different designs and gain deeper insights into user performance and experience. Those metrics are...

Task Time

This measures how long it takes participants to complete tasks. Tracking this helps us identify inefficiencies and streamline the process.

Average Errors Per User

This tracks the number of errors made during usability testing. By analyzing where and why errors occur, we can improve the user experience.

Single Ease Question

After each task, participants rate the ease of use on a 1-7 scale. This feedback helps identify specific areas for improving task usability.

System Usability Scale

At the end of the test, participants rate overall usability and learnability on a scale of 0-100, offering broader insights into the product experience.

What We Learned from Usability Testing

Creating or Editing Fees Use Case

During the usability tests, we evaluated two different designs for the “Creating a Fee” and/or "Editing Fees" use case. The Inline design allows users to click a button, and a new row appears within the table, enabling them to input fee details directly. In contrast, the Sidesheet design opens a side window on the right when the user clicks a button, guiding them through a step-by-step process to create the fee.

Inline Layout

Sidesheet Layout

The graphs below depict data on creating fees for two different layouts: inline and sidesheet. Over four weeks of testing, 18 participants tested the inline layout, while 27 participants tested the sidesheet layout..

The data clearly shows that the inline layout significantly outperforms the sidesheet in both user performance and sentiment.

The average task completion time for the inline layout is faster (92 seconds) compared to the sidesheet (113 seconds)

The total number of errors was much higher for the sidesheet layout, with 41 errors recorded, compared to only 14 errors for the inline layout.

The error rate, shown is much lower for the inline layout (0.8 errors per user) than for the sidesheet (1.5 errors per user),

Lastly, users also expressed a higher ease of use with the inline layout, with an average SEQ score of 6.6 versus 5.9 for the sidesheet.

Disbursing Fees Use Case

For Disbursing Fees, the following data shows how the design evolved over the weeks based on user feedback and testing insights. Across weeks 1, 3, and 4, a total of 24 participants were involved, with 8 participants tested in Weeks 1, 7 participants in week 3, and 9 participants in week 4.

Week 1 and 3

In the Week 1 and Week 3 design for the Disbursing Fees use case, when a fee was ready to be disbursed, it would automatically populate in the “Staged” table. From there, users could either disburse fees individually or disburse all fees at once by clicking the “Disburse All” button. While the design remained the same in both weeks, the participants were different between Weeks 1 and 3, providing fresh insights each time.

Step 1 - Select Fee(s) to be Disbursed

All fees ready to be disbursed will populate in the “Staged” table. The user then clicks “Disburse All” to send all fees at once, or selects an individual fee to disburse separately.

Step 2 - Verify Fees to be Disbursed

User reviews the fees to ensure all details are correct, including amounts and recipient information. Once verified, the user proceeds to send the fees for disbursement.

Qualitative Feedback

In Week 1, users found the disbursement process easy, but the new “Staged” table caused some confusion as it wasn’t part of the legacy design. 4 out of 8 participants found the disburse process simple and time-saving, while others struggled to locate the disburse function in the kebab menu. Additionally, 5 participants wanted more information in the modal window to double-check fees before sending them out.

In Week 3, 5 out of 7 participants described the disbursement process as “easy,” but similarly requested more details in the modal window to verify fees before disbursing.

“It’s straightforward and cuts time for funding, but I’d like more details in the window to make sure everything’s correct before disbursing.”

- Funding Associate (Week 1)

“The process is great once you figure it out, but the ‘Staged’ table isn’t immediately clear. It took me a moment to understand where to find the disburse option.”

- Funding Associate (Week 3)

"Once you figure out the ‘Staged’ table, it’s easy to disburse fees, but it takes a moment to understand how it works at first."

- Funding Associate (Week 3)

Week 4

After gathering feedback from Weeks 1 and 3, we made several key changes to the disburse feature design. These adjustments addressed user concerns, such as adding more information to the modal window for verification and improving the visibility of the disburse function, ensuring a smoother and more intuitive process for users.

Change #1: Removed "Staged" Table

One of the main changes was removing the “Staged” table and moving the “Disburse All” button to the top right of the screen to reduce confusion. A “Staged” status now appears in the status column to indicate when a fee is ready for disbursement.

Change #2: Added a Verification Method

The second change introduced a way for users to verify payee information. In the disburse modal window, users can expand a row to review the details before clicking “Disburse All,” ensuring accuracy.

Quantitative Data

The average task time in Week 1 was the highest at 130 seconds, as participants struggled to understand the new flow, specifically the “Staged” table. By Week 3, after users had become familiar with the process, task time decreased to 67 seconds. However, in Week 4, when the design was updated, users had to adjust to the new changes, leading to a slight increase in task time to 75 seconds.

The SEQ score reflects ease of use. In Week 1, the average SEQ score was 5.3, indicating that participants found the task challenging due to the new elements, such as the “Staged” table. By Week 3, familiarity with the design increased the SEQ to 6.9, showing improvement. In Week 4, the SEQ reached a perfect score of 7, as users appreciated the changes despite needing to relearn aspects of the flow.

In Week 1, participants made 10 total errors, primarily due to confusion around the new “Staged” table. By Week 3, errors dropped to just 1, reflecting a greater understanding of the process. However, Week 4 saw a slight increase to 4 errors, as users had to adjust to the changes made based on previous feedback.

The error rate started high in Week 1 at 1.25 errors per participant due to the learning curve of the new features. By Week 3, the error rate significantly dropped to 0.13, showcasing how quickly users adapted. In Week 4, the error rate rose slightly to 0.7 as users adjusted to the design updates.

Achieving Success Through A Mixed Method Approach

05

After incorporating key insights and recommendations from previous iterations, we developed a final design that successfully addressed user pain points and streamlined the fee creation and disbursement process. This version aimed to reduce complexity, eliminate unnecessary steps, and provide users with a more efficient and intuitive workflow.

Despite the improved performance of the inline layout, we collaborated with engineers to meet in the middle by creating a sidesheet design that displayed all necessary information in one view, rather than page by page. This approach combined the efficiency and clarity users loved about the inline layout with the speed and ease of development of the sidesheet.

Locate the desired section and click “Add Fee”

Creating a Fee(s) Workflow

Add Fee Information & Documentation

Optional: Add Second Fee

Disbursing a Single Fee Workflow

Select Desired Fee & Click Kebab Menu

Verify Fee Information and Click "Disburse"

System Verifies Fee Has Been Disbursed

Disbursing a Multiple Fees Workflow

Click "Disburse All" Button

Verify Fee Information and Click "Disburse All"

System Verifies Fee Has Been Disbursed

The chart shows the steady improvement in user sentiment as we iterated on the design. In Week 1, the SUS score was 74.5, reflecting initial user frustration.

By Week 3, after incorporating usability feedback, the score rose to 86.5, and it peaked at 92 after further refinements.

Although Week 4 saw a slight dip to 89, the overall trend highlights significant improvements in usability and user satisfaction through continuous iteration.

System Usability Scale Progress Over 4 Weeks of Testing

Reflecting on Key Learnings

05

Reflecting on Key Learnings, the mixed-method approach was crucial to the success of the project.

Contextual inquiries allowed us to see how all three main user groups—Closing Associates, Pre-Funding Associates, and Funding Associates—used Pinecone. This provided valuable insights into their workflows, enabling us to design a single page that consolidated all necessary functionalities. It also highlighted key pain points we could address through iteration.

Usability testing was equally important, as it allowed us to refine the design week by week, ultimately creating a solution that was easy to learn and use for all user groups.