Background
The Veracode platform is comprised of several different application security scan types, each functioning semi-independently. I was the lead designer for Software Composition Analysis (SCA), working closely with two teams of engineers and two Product Managers.
Previously, Veracode had acquired a start up that provided a similar SCA service. It was loosely combined with their existing SCA offering, but the two systems were not well-integrated.
This project was part of the larger on-going effort to determine the future positioning and shape the functionality of our SCA tools to deliver more value and a better user experience.
Problem Statement
Most customers aren’t realizing the full value of our product, negatively affecting retention & renewal rates. Many customers who were licensed for this new product weren’t using it at all.
Goal
Better understand the most valuable aspects and pain points of each offering to help inform a unified product strategy, identify opportunities for better integration, and remove barriers to adoption.
Users & Needs
The product serves two main user groups: developers working to write code that meets the security standards of their organization, and software security professionals who create those standards and manage their organization's security program.
Broadly speaking, developer users are looking for clear information about what needs to be fixed, why, how, and by when. Security managers need administrative and analytical tools to define their program goals and monitor progress and adherence over time.
Mapping the Current State
The current state offered two parallel paths with similar functionality but significant differences in details & execution along the way.
The goal was to identify the most valuable aspects and pain points of each offering to help inform a unified product strategy, uncover opportunities for better integration, and remove barriers to adoption.
Key Questions:
- Where can objects and processes be consolidated?
- What distinctions are valuable to retain?
- What are the barriers to adoption for product B?
User Research
We started our research with 1:1 interviews with 8 internal customer support engineers and solution architects who have close contact with customers and how they use our products. This allowed us to better understand customer use cases and get an overview of common problem areas.
We then used these learning and preliminary ideation on possible future states to create a script for in-depth customer interviews. With the help of PMs and account managers, we secured interviews with 7 customers during the fall of 2020, which included a range of organization sizes and a mix of current users of either or both products.
Findings
From this research emerged a salient image of a fractured user experience moving between two siloed workflows, loosely bound together as a single product. Users struggled to find effective ways to use these different aspects of the product together and in concert with other products in our platform. The disparate organizational structures & hierarchy, and to a lesser extent, nomenclature and visual presentation, posed a significant learning curve for existing customers or anybody moving between the two systems. While this was not new information, the power of words and anecdotes directly from our customers helped spur action to address these concerns, and the details we gathered provided guidance for how we could do so. Looking at the major areas of dissonance, we identified policy as a good starting point, based on a combination of technical feasibility, our confidence in our understanding of the issues and the proposed solution, potential for positive impact to the customer, and laying necessary groundwork for future initiatives.
Goals
Design a unified policy management experience as a first step toward complete unification between the two product experiences, including:
- Augmenting the existing SCA policy rules to incorporate all desired functionality
- Allowing users to apply those policies in both scan workflows
- Reducing duplicative work and upfront configuration
- Balancing granular control with ease-of-use
Feature audit of the two systems to be combined
Before
Pain points:
- Long-established user difficulty using Controls
- Basic discovery, may assume governed by a default policy rather than a different system of rules
- Irregular multi-step process to enter editing mode
- Vulnerability severity vs. Issue severity caused confusion, poor visibility/summary view
- Duplication of work for ranges, with/without vuln method
- Goal = provide support to create logical tests, but result was jargony and overly complicated
- Legacy constructions that no longer made sense, e.g. “Create Issue” action is only option remaining > eliminate
- Unclear how conflicting controls will be resolved
After
The unified policy designs represents a major increase in usability and efficiency. Instead of needing to translate and rebuild their policy from one system to another, users can now create a single policy for use by both systems.
Highlights:
- Significant efficiency gains
- Leverages existing patterns to introduce established best practices
- Each rule type is self contained with clear logic and purpose
- Wizard guides users through the configuration process
- Clear high level view of rules & conditions, both during editing and after creation
- Visibility into which other applications are using each policy
- Augments existing rules to include essential functionality identified from the deprecated system to carry forward
- Optional complexity with smart defaults and simple vs. advanced views
- Streamlines logic for license handling
- Centralizes on single measure of severity to reduce ambiguity and align with the industry standard
Policy Summary
New Rules
Expanded Rule
Edit Policy
Explorations
License Risk
