This guest post was written by Jordan Billingsley, the BIM Coordinator and Design Technology Leader at Hord Coplan Macht. In the post, Jordan describes the method he used to assess software usage and skills at his firm. Take it away Jordan!
Is your software super or superfluous?
Software is a major investment for every design firm. In the era of the “app”, “monthly auto-renewal”, and “freemium”, it’s easy to accumulate a variety of software, from stand-alone applications to add-ins to mobile software.
Determining the effectiveness of all that software can be a daunting task. Sure, separating the wheat from the chaff is easy. But determining the most useful, while also respecting user preferences, is infinitely more difficult.
My firm recently undertook an effort to measure the effectiveness of our software investment. To do so, we used a combination of online surveys, user interviews and cost analysis. As a result, we were able to map our firm’s software use, develop an employee technical skills matrix and target our training efforts.
Here’s are 5 steps you can use to conduct your own software and skills audit:
Step 1: Take a Software Inventory
If your firm has a dedicated IT department, they will most likely have monitoring software in place. Ask them for a list of all the applications in use.
If your firm doesn’t have an IT department, check out Belarc Advisor. It quickly creates an inventory of all the hardware and software on your network
Step 2: Conduct a Survey
Once you have an inventory of all your software, use that information to survey your users on what software they use regularly and how they feel about it. Surveys are easy to implement but don’t always produce good data. Here are some tips to getting the most out of a survey:
- Use Typeform, SurveyMonkey, or similar software to create simple, well-designed forms that are easy to fill out.
- Make the survey short and easy to complete. According to Survey Monkey, abandonment rates increase after 7-8 minutes. Multiple choice questions are easier to answer and analyze.
- Conduct one survey at a time. Don’t bombard your office with other surveys. Also, set a short deadline. Three to five days is best.
- Be sure to describe the importance of the survey and the individual impact of each user’s input.
- Send the survey with an endorsement from a firm leader or decision maker.
- Incentivize participation with guaranteed prizes or raffle entry
What does a well-designed survey look like? Click here to download the survey I created for my firm.
Step 3: Ask the Hard Questions in Person
Surveys are a good starting point but can only go so deep. Follow up the survey with some in-person interviews. This is also a good way to identify the hidden software experts in your firm. Some questions to ask are:
- What software is being underutilized? Why?
- When did you last look at possible alternatives?
- Are workarounds necessary? Are you using a complicated workflow to achieve the final result with the current software?
- Is there an industry convergence on a particular tool? Are previously specialized applications becoming features in mainstream platforms?
- Are updates occurring frequently? How is the user community? Is it growing or becoming stagnant?
Be sure to take notes and record their responses on a specific form. This data will help fill in some of the gaps from the survey responses.
Step 4: Take a Hard Look at the Costs
Now that you have input from users, it’s time to look at the numbers.
The soft and hard costs for each application may have changed since you first purchased the program. It is important to look at available pricing tiers, enterprise or floating license options, and the software’s computing requirements.
If the software is outpacing your hardware replacement cycle you may want to either a) stop software updates b) recalculate depreciation costs or c) consider alternatives to dedicated workstations such as mobile workstations or shared virtual machines.
Step 5: Analyze and Decide
Take a good look at all the data you collected. For each application, use a grading system to determine which of the 3 Rs should apply; remain, remove or replace.
Get rid of all the software marked as “remove”. It’s clearly no longer needed and is only taking up space.
Double-down on software marked as “remain”. These are your core applications. Invest in training and refine your standards and workflows.
If an application is marked as “replace”, the prospective replacement software should be evaluated by an in-house expert. Use a standardized evaluation form to help guide the process and result in a uniform reporting format. Click here to download a sample software evaluation form.
These evaluation reports may be helpful in the future when testing similar software for more accurate comparisons. If replacement software is approved for trial, thoroughly test with core users first and develop internal documentation before making a formal announcement of any software discontinuation. Replacing software isn’t always an easy process. There should be significant overlap in the software transition period.
What We Learned
Right off the bat, our IT department eliminated three inactive applications. These applications were not incurring any direct expenses but were taking up server space and receiving maintenance. Our IT department also helped identify 18 of the most heavily used programs to form our survey criteria. We chose to not include ubiquitous titles such as Microsoft Office.
We next focused our attention on the self-assessment responses. Those who marked themselves as a 4/5 or 5/5 were considered “experts”. We then used these results to tackle a long-standing to-do-list item: create an Employee Technical Skills Matrix.
While self-assessment may not be the most accurate measurement of skill it gave us a good starting point. The goal of the matrix is to help build balanced teams with the appropriate experts. We also built-in Professional Development Goal Tracking to identify aspiring new hires. This feature provides the ability to create more natural shadow training. If someone wants to learn 3D printing, we can pair them with a Fabrication Specialist.
With our experts identified, we were able to gain additional insight from our data interpretations. This was especially helpful when analyzing responses to the question “What Software is Under-Utilized?” The unfiltered data suggested that our hot new software Lumion warranted additional training. After looking at responses from our experts we saw Lumion drop and Dynamo rise in ranking. The first assumption was that experts valued automation abilities over rendering skills. This was disproven after conducting follow-up interviews.
The limitation resulting in under-utilization of Lumion was simply access to advanced hardware. Our experts have priority access to laptops with gaming graphics cards. This explained the shift in the ranking. Once a user had access to the appropriate hardware, the existing training materials allowed them to make full use of the software. Dynamo, on the other hand, was recognized as extremely useful but remained underutilized due to a lack of training materials and time.
The cost savings of this software audit are ongoing as we evaluate prospective replacements. However, there have been some immediate expense reductions. A merger with another office resulted in some duplicate subscriptions that have since been consolidated. We were also able to avoid the purchase of additional new software after connecting identified experts with users requesting a specialized application. As it turned out, some of our current software already had a feature that met their needs.
This evaluation process has also helped prioritize our goals and form a more convincing statement to firm leadership. We can now clearly demonstrate who are experts are, what employees value, where we should concentrate our resources, and why all of this is worth doing.
How About You?
Have you conducted a software and employee skill audit to measure your software investment? If so, how did you go about the process? What did you learn from the results? Leave a comment below!