Some people might think paying several hundred or thousands of dollars a month for an AI Agent for sales / support is a bit much.

We get it.

It's a relatively new industry.

But we believe that when implemented well for a sufficiently sized business their ROI can be 3x to 10x. And we're so confident in that proposition, that if within 60 days of signing with us if we can't at least 3x your ROI we're willing to give a 100% refund!

But, how will we measure if we've at least 3x-ed your ROI? Short answer: We can set-up AB testing for you. Longer answer: Please keep reading.

Our Step by Step Process

Step 1. Defining your button

As Kevin Hale of YC fame mentions, the goal of any website is to get people to press a button.

This typically is a button that could lead to:

• a sign-up
• a meeting booking
• a lead form submission

In our experience, we've seen an AI Sales / support agent implementation done well could lead to a 15%-30% improvement in that button getting clicked.

How?

• It helps people quickly find the info that if they hadn't, might have led to them disqualifying your offering as an option.
• A good AI agent nudges high-intent users to leave their contact info

Step 2. Quantifying how much money you make / lose each time someone clicks the button

This isn't as complicated as you might think.

You just need 3 things:

1. Get a sense of how many clicks that button gets in a given time period … say, a week. This can be fairly easily obtained by Google Analytics <> Google Tag Manager or another analytics tool of choice.
2. Find out what percentage of those who click that button end up as paying clients. For eg. You may find out that 1% of those who click a sign-up button actually end up converting to paying clients.
3. Finally, you need the LTV (or Lifetime Value) of a typical customer. This can be fairly easily obtained from Stripe or calculated by some variation of the following formula LTV= Average Profit or Revenue Per Account / Churn Rate. Alternatively, you can calculate how much it costs you on average to deal with a service request each time someone clicks that button.

Once you have the above data, you'll be able to say something like:

I know that in the past month, I had 1000 people clicking the sign-up button, 70 of those converted to paying and the LTV on average of each of those is $100. i.e. 1000 clicks on my CTA are worth 70 x 100 = $7,000 to me, so one click is worth $7 ($7000 / 1000 clicks).

Or, 1000 people clicked the request service button last week and each click on average cost my service team 10m to service, with a $30 / hr rate that's 10 / 60 x $30 = $5 / request.

Step 3. Do a sanity check

In an implementation done really well, we can expect a 30% gain. So, assuming the above numbers are true for you a well-implemented AI agent might be able to take you from 1000 clicks to 1300 clicks. Given that the new 300 clicks on average get you $7, the potential upside for you is $2100 / month.

Since there's always risk and overhead, it's good practice not to pay more than 1/3rd the potential gains so the most you should be willing to pay for an AI agent implementation is $2100 / 3 = $700 / month.

If the above number is something modest, like $100 / month, we recommend you search the market for a low-cost $20 / month type solution since the added customization benefits do not yet make sense at the current stage of your business. That will likely still give you some gains in improved conversion rates.

Step 4. Setting up an AB Test

If you're continuing to read, we're assuming that you have a gut feel that a 15-30% improvement in conversion rate on your landing page could be worth several thousand dollars a month. Now, let's get you some data. Or, a similarly sized reduction in support tickets could have that much saving.

i. Choosing an AB Testing Platform

For this example, we'll be using posthog. Though, you're welcome to use any AB testing tool of choice.

ii. Setting tracking for 'button clicks'

Now this is counterintuitively hard.

a. The Naive Way: Setting up custom tracking on the relevant button to see how often clicked

What happens in AB testing is that a portion of the audience is shown a variant A (say no ai agent) and another portion is shown variant B (ai agent present). Then the goal is to see if variant A or variant B led to more 'button clicks' of whatever meaningful button you're tracking.

In most ab tests, people set-up a custom event to fire each time the given button is clicked. (how to do that in posthog).

The problem of doing that here is that this doesn't capture clicks on that button 'inside the AI agent. For instance, it's not uncommon for AI agents to have a main CTA button inside (ie 'Book a demo' button below). Those clicks won't be captured in this kind of set-up meaning the results can't be trusted.

b. The comprehensive way: Use an EventListener to trigger success tracking

To explain, let's take the example of chatsimple.ai's landing page.

Say our most important button is the new 'Sign-ups'. Anytime someone clicks 'Sign Up' or 'Get my AI Agent' or an equivalent button on our landing page, they get re-directed to app.chatsimple.ai (see below).

The first thing we have to do here is add an exclusion for the log-in page. See the address bar below. It has '?log_in=true' at the end, we had to add that so we can exclude from tracking all the folks that are just coming back to log-in (a healthy chunk of clicks on the page).

Once that's set-up, we can add an eventListener that fires when anyone navigates from the chatsimple.ai to the app.chatsimple.ai page where log_in=true absent, and records that as a conversion event using posthog.capture(). See below.

"use client";

import { usePostHog } from "posthog-js/react";
import React, { useEffect } from "react";

const BodyScripts = () => {
  const posthog = usePostHog();

  const trackSignUpConversion = (url) => {
    if (
      url.includes("https://app.chatsimple.ai") &&
      !url.includes("log_in=true")
    ) {
      posthog.capture("converted_to_sign_up");
    }
  };

  useEffect(() => {
    const handleClick = (event) => {
      // Get the deepest element that was clicked (including shadow DOM)
      const path = event.composedPath();

      // Check each element in the path for an href
      for (const element of path) {
        if (element instanceof HTMLElement) {
          if (element.hasAttribute("href")) {
            const href = element.getAttribute("href");
            trackSignUpConversion(href);
            break;
          }
        }
      }
    };

    // Use { capture: true } to catch events in the capture phase
    window.addEventListener("click", handleClick, { capture: true });
    return () =>
      window.removeEventListener("click", handleClick, { capture: true });
  }, [posthog]);

  return (
    <div>
     {/* Chatbot embed scripts to be added here later */}
    </div>
  );
};

export default BodyScripts;

iii. The last bit is setting up feature flagging (ie the real ab test)

This allows the ai agent to only show-up half the times (or another number of choice)

This is fairly easy. See below how we've done it below using posthog.

"use client";

import { usePostHog } from "posthog-js/react";
import React, { useEffect } from "react";
import Script from "next/script";

const BodyScripts = () => {
  const posthog = usePostHog();

  const trackSignUpConversion = (url) => {
    if (
      url.includes("https://app.chatsimple.ai") &&
      !url.includes("log_in=true")
    ) {
      posthog.capture("converted_to_sign_up");
    }
  };

  useEffect(() => {
    const handleClick = (event) => {
      // Get the deepest element that was clicked (including shadow DOM)
      const path = event.composedPath();

      // Check each element in the path for an href
      for (const element of path) {
        if (element instanceof HTMLElement) {
          if (element.hasAttribute("href")) {
            const href = element.getAttribute("href");
            trackSignUpConversion(href);
            break;
          }
        }
      }
    };

    // Use { capture: true } to catch events in the capture phase
    window.addEventListener("click", handleClick, { capture: true });
    return () =>
      window.removeEventListener("click", handleClick, { capture: true });
  }, [posthog]);
  
  
  // New code that sets up the feature flag
  const getChatbot = () => {
    const tag = posthog.getFeatureFlag(
      "check-conversion-rate-for-chat-bot"
    );
    if (tag === "test") {
      return (
        <>
          <co-pilot
            platform-id="06c36c1csdf531-c869753d4346"
            user-id="5955914-dsf23423fsdf-frge3434"
            search-ai="true"
            chatbot-id="13fd518b-a832-46e0-a946-605823423488f90"
            base-url="https://chatsimple.ai"
          />
          <Script src="https://cdn.chatsimple.ai/ai-loader.js" defer />
        </>
      );
    } 
    return null;
  };

  return (
    <div>
      {getChatbot()}
    </div>
  );
};

export default BodyScripts;

If a website has traffic of 10k+ visitors a month. Getting statistically significant results should be a week or two wait. Possibly sooner.

Follow-up considerations

It is possible that sometimes an ai-agent might decrease the landing page conversion rate because it helps unqualified customers gauge faster if they're not a good fit.

That practically means a lower conversion rate on the landing page, but one step down the funnel there's a higher conversion rate (say more people convert to paying) leading to a net gain in revenue still.

Therefore, keeping an eye on the rest of the conversion funnel during testing is best practice.

For some of you, the above hints might be enough to go on. In which case, glad to have helped! For others, you may want some more help setting up the above. In which case, we have an offer for you.

Chatsimple's Offer

If you're willing to give us a shot by signing up for a Business plan or higher, then we're willing to do all the above AB testing set-up for you at our own expense. If your website has niche edge cases, we'll have a devoted engineer jump-in to set-up whatever custom tracking is needed.

The above example was sign up button clicks, but it's possible to do a similar set-up for lead capture, contact form fill outs & so on.

Then within 60 days, if we're not able to quantitatively show that we've at least 3x-ed your ROI we'll give you a 100% refund.

Exclusions

To make sure our offer isn't abused, we only make this offer to select businesses.

Do you qualify?

The best way would be to book a call with us to find out.

But generally, we look for businesses:

• For whom a 15-30% improvement in conversion rate is worth several $1000 / month.
• They have some variation of analytics tracking system set-up to calculate ROI or are willing to accept our help in setting up.
• They show a willingness to commit to bi-weekly meetings in the first 60 days of the engagement to iron out any kinks.