February 24, 2021
How to Evaluate DevOps Solutions

How to Evaluate Salesforce DevOps Solutions?

DevOps as we know it, is not a set of tools, a team, or a role, rather it is a mindset. Today we discuss how you can properly evaluate native Salesforce DevOps solutions to ensure that you’re making the right decision for your organization, culture, and processes.

5 WAYS HOW NOT TO EVALUATE DEVOPS SOLUTIONS

Evaluating DevOps solutions is easier than you think! It comes with its own set of challenges and thus, we often see decision-makers making the wrong choice. So before we discuss how to evaluate DevOps solutions, here are some ways that decision-makers fail to properly evaluate a DevOps Solution:

  • Don’t let a marketing pitch sway you:

Marketing razzle-dazzle is one thing but don’t let it waiver you from what really matters. Instead of the marketing blitzkrieg, focus on the amount and quality of innovation as well as the success that the solution has delivered to its customers. Also, during evaluation don’t be afraid to ask for and consider the customer retention ratio and whether it offers customers sufficient customization.

  • Don’t make your decision on the first date:

A lot of customers fall in the love at first sight with the demo and make their decision on the first demo itself. Instead, make the effort to understand the solution with due diligence, and ensure that it meets your evaluation matrix, performs on all metrics, and has the functionalities to meet your business objectives.

  • Don’t begin the evaluation process without a lack of clarity on detailed outcomes:

Begin with the end in mind. It helps you know the technologies, features, and functions you need to get there.

  • Don’t hide facts, transparency wins:

Ask the solution vendor on ways they will benchmark performance vs. other solutions. Putting your challenges and concerns upfront will help the solution vendor offer a detailed and customized solution. These together can help make decision making fast and easy.

  • Don’t skip rigorous security evaluation:

Involving your security team early in the process ensures that the solution confirms all the compliances and security measures. Similarly, don’t just blindly go by the compliance certifications that the solution vendor may claim to have. Instead, dive deep to ensure that the architecture of the solution ensures that the solution is indeed secured and does not violate data security rules.

How to Evaluate Salesforce DevOps Solutions

When evaluating a Salesforce DevOps solution, your entire journey can be broadly broken down into three aspects. The article further highlights the steps you must take at each and every evaluation phase:

Step# 1 Before the Evaluation Process

Step #2 During the Evaluation Process

Step #3 Taking the Final Call

Step #1 Before the Evaluation Phase

First things first, the key is to not confuse the tools of deployment with processes. If you want to speed up deployments, then work on speeding up the deployments with automation and you can later work on DevOps or Agile. Choosing CI/CD gives you shorter deployment windows and faster release cycles.

Strange but true; when we talk to the customers, most of them have little clarity on why they need the solution. Begin with the end in mind to discover your ‘Why’. If you don’t know where you are going, any road will get you there!

If your team is not clear on the objectives you want to accomplish with the solution or the challenges you wish to meet, there are high chances you may not necessarily choose the best solution. Here are some common objectives that our clients look to achieve with a DevOps solution:

  • Automation in deployment/ testing
  • Simplified Code/ configuration merge and review
  • Code configuration linked to the value
  • Determine vendors (Separate the Wheat from the chaff)

Simply put, there is no point wasting your time and effort evaluating 10 odd solution vendors rather it is recommended to narrow down your final shortlist to 3 or 4 vendors. A great way to handpick vendors is to validate and look for reviews on Salesforce AppExchange and G2 Cloud.

A key deciding factor is to understand whether the Salesforce DevOps solution is the major product offering of the vendor or just one among their many products. It determines the support and innovation you can expect in a solution in the near future. Next, you may look at other key factors such as

  • Their alignment with the Salesforce platform
  • The innovation they have delivered to their clients.
  • Do they fall short on innovation, are simply using open source and standard version control system or scrips?
  • How is the vendor addressing common challenges and the nuances of the Salesforce platform in the context of the app release process?

Here are a few tips to keep in mind as you go through the first step of evaluation process.

  • Understand the Why? What you want to accomplish?

Get a firm clarity of your objectives. Answering the ‘Why’ before the what and how is key to finding more clarity and direction to find a DevOps solution that fulfills your business objective. It will give you what features you need to look for to solve the problems or challenges in Salesforce DevOps. As mentioned, knowing what it is you are looking for will help you get there faster and successfully.

What challenges are you looking to solve as well as the timeframe? The maturity curve of most customers reveals that they use a certain portion of the DevOps solution later. For instance, most initially focus on deployment, code mergers while usually leaving regression testing for later. Clarity on what is it that you want to accomplish will help you chalk out specific objectives to make decision making easy.

  • Set up an internal cross-functional team for better evaluation

It is best to handpick a cross-functional team of Salesforce developers, admins, and release managers so you have representation from each of those roles. Having all of them attend all the meetings will allow them to give their unique insight to help you pick just the right solution.

  • Include the security team and compliance officer at the outset of the evaluation process

Security can no longer be an afterthought, either in-app deployment or in the evaluation process of a DevOps solution. It may so happen that the DevOps solution that you think best addresses your concern may not have sufficient security quality gates in place and will eventually be vetoed by the security team. Having a security team as an integral part of the evaluation process can help you take better decisions faster. Similarly, having a compliance officer on board the DevOps evaluation team too will help you address specific compliance-related requirements upfront as well as concerns regarding producing compliance certificates.

  • Establish Timelines and milestones

Having an understanding of the timeline for the start of implementation and the progress of implementation is critical. Based on your internal project plan, you can reach out to the vendor for more specific requirements as well as a more customized project.

  • Determine high-level alternatives

Determine the high-level options you want to go with. For instance, do you want to use Salesforce DX? Have you spoken to other customers who have adopted Salesforce DX to know their views to help you better understand how you can fully leverage it quickly?

  • Grade the vendor to help you shortlist them better

You want your Salesforce admins and developers to focus on high-quality tasks, building new features rather than simply moving the changes across. Grading vendors on their ability to innovate and how their solution is unique will help you choose a better strategic choice. Equally important is the vendor’s capability to offer 24×7 support with a high response rate for your team spread across the globe.

# Step 2 During the Evaluation Process

As you move forward in the next stage of evaluation process, your evaluation team must ask to attend a demo session to understand how the DevOps solution will truly solve the pain points and fully leverage the platform. But there’s a catch! Here are few tips to help you choose the Salesforce DevOps solution for high ROI.

  • Insist on a demo session

Don’t let the fancy slides and graphics distract you from the fundamental purpose of choosing a DevOps solution that solves your list of pain points you are looking to solve. Look out for firstly, which feature solves which problem for which problems is the solution solving and secondly on the outcomes that are how the solution will ensure that the quality of your releases is better.

The slides are a tool so your focus must be on the functionalities and features that the product has and the vendor offers not how it will help you achieve your outcomes. Try to understand how the demo meeting can achieve the outcomes. Focus more on the functionality that they showed rather than just what they claim to deliver. You must look at what are the features and how they will help you solve the pain points that you are facing

  • After the demo don’t forget to measure and analyze

While the first demo will likely give you a teaser of what all the solution can do, it is simply not enough. Therefore, it is recommended to schedule yet another detailed demo along with the DevOps team to better understand the complete end to end flow in action, right from pulling the changes from the development boxes, committing those changes to user story to merging of code among different developers, doing the code review process and delivering them into higher orgs. and user stories as well as deploying changes, etc.

A single demo session is barely enough for you to really understand what the solution can door slide determine your decision to choose the solution. Moreover, during this phase, you must care to measure

  • The time taken to complete all the changes and operations
  • What percentage of transactions slides did you see live?
  • Were the setups shown live? Did you have complete end-to-end visibility or was it a black box behind the scenes?

This helps you thoroughly assess if the solution does indeed meet your requirement and whether you should proceed further to the next stage of the evaluation process.

#Step 3 Taking the Final Call

The stakes are high and you certainly don’t want to make the wrong choice in choosing the DevOps solution. Here are three key aspects to consider to avoid making the wrong choice:

A,B,Cs to Avoid Making the Wrong Choice in Salesforce DevOps

A) Chalk out a broad matrix to get a bird’s eye view:

As you move towards zeroing in on the best-fit DevOps solution, it is highly recommended to get a bird’s eye view of the challenges you face, capabilities listed by the vendor, and the results you can expect out of the solution with a simple matrix. For instance, if overwriting code is a major challenge, what are the features that the solution introduces, and what is the result you would get as part of the process.

B) 5 key questions for better security evaluation:

Security and compliance are important and should be part of the evaluation criteria as well. Shockingly most customers often skip this or don’t ask for it. Although when evaluating the Salesforce platform most companies thoroughly analyze it from a risk and security perspective, but when it comes to evaluating a DevOps solution, we often see that many customers skip the step even though it deals with the same data that a Salesforce platform is dealing with.

Here are 5 questions you need to ask so you know that the DevOps solution has you covered:

Q#1 Will the vendor have backdoor access to your production org?

Do you have open IP addresses during installation? Even as customers are using products like Salesforce Shield to protect the data from internal employees, they are giving access to their entire data set to the DevOps vendor. Choosing a native DevOps solution like Flosum ensures vendor does not have access to your production org.

Q#2 Will the vendor move your business data on their infrastructure? Are you okay with losing control of your data?

Are they asking you to sign data protection agreements? Depending on your industry, this may be a potential threat to your security. As you look at the newest solution or opportunities, you should not forget the fundamental idea that creates a full-spectrum solid cybersecurity foundation and cyber-resilient systems for your defense, intelligence community.

Q#3 Does the vendor use a compliant infrastructure? Does the usage make it compliant?

An equally important factor is in the context of compliance. So, although the underlying platform may be compliant, the important question to ask is whether the vendor too is compliant with that platform. So if you are a healthcare organization seeking HIPAA compliance or PCI compliance choosing a DevOps built on Heroku, it is not compliant by default unless you go through the certification process.

Q#4 What are your data residency requirements?

There are certain customers and countries such as Canada, Germany, Japan, that require that the data should not leave the country, especially the consumer data stored in the Salesforce instance. Thus, it is important to ensure that the vendor will not move your data to a different country or it does not even leave the Salesforce instance.

C) Measure 3 Key Benchmark Metrics

  • Transaction Time (Benchmark with use cases)

Data doesn’t lie. Therefore, benchmarking metrics for certain use cases for all the shortlisted solutions is a great telltale sign about its efficiency. Simply take generic use cases like deployment, rollback, deploying profiles as well as certain complicated use cases and do the benchmark performance for different DevOps solutions to assess which one will help you meet the goals faster, hassle free and take the stress off the developer.

Test them through code conflicts, overwritten changes and even deploying things out of sequence and analyze how long it takes to move the changes from dev to production. That will be a clear indication of how the DevOps solution is built and how it can handle all those usual problems.

  • Time to Value

How long does it take you to do the first deployment and bring change in the quality of releases as well as delivery velocity? What is the time to value once you install and configure it? Does it take months? Weeks? Days or just minutes? For instance, with Flosum, customers can configure everything out of the box and get the first deployment done, literally within the first one hour. Equally important is the time it takes for full configuration. If you want to integrate it with Jira, or other systems, what is the time taken to integrate and configure it with other systems as well?

  • Training Time: Time to onboard a new employee on the solution

We often see in the Salesforce world that Citizen developers, admins, and business users make changes to the Salesforce platform. So, if you want to democratize your application development and enable more business users to make more changes to your Salesforce platform, this metric is critical. Choosing a native Salesforce DevOps solution offers the fastest onboarding time since it leverages Salesforce’s familiar UI for all actions.

Other Key Criteria to Consider

  • Is the app listed on Salesforce App Exchange?

If it is listed, how many reviews does it have and how good are the reviews. Also, look for 10 to 15 reviews to get a diverse opinion and be on the lookout for suspicious reviews that may have been actually promoted by the vendor. When was the latest release? If the latest release was quite a few months ago, it might have unresolved bugs.

  • Is the app too bulky? Do you have to install multiple systems/ platforms?

Salesforce was designed to be lightweight and agile and deliver fast. But if the solution is bulky and requires multiple systems, the question you must ask yourself is if it is worth it. Wouldn’t you rather prefer a DevOps solution that is lightweight, fulfills all the objectives, and get the work done faster too.

  • How deep is the partnership with Salesforce?

Do they often work with Salesforce? How well do they leverage the strength of the Salesforce platform and native strengths?

Your Search for Native Salesforce DevOps Ends at Flosum!

Choosing the best fit Salesforce DevOps solution is no easy task and it can be hard to accurately judge what tools you really need. Flosum makes it easier for you.

Flosum is a complete end-to-end 100% native solution including built-in merge tools, version control, continuous deployments, static code analysis, user story management & regression testing tools. It offers numerous benefits over other existing Salesforce DevOps solutions, thus making it the most trusted all in one DevOps solution for Salesforce.

Ready to simplify your Salesforce Deployment?

Next Steps