It's Monday -- that means it's time to open up the mailbox and answer YOUR questions!

It’s Monday — that means it’s time to open up the mailbox and answer YOUR questions!

Sometimes you’re just a little tweak away from a big winner…

Rainmaker, today’s Mailbox Monday hits on one of the topics that got me most excited when I was discovering direct response marketing…


Specifically, how you can try a couple different approaches, figure out what the market responds to more, and sometimes double, triple, or 10X the response to — and profitability of — your marketing campaign.

In fact, my very first client, David Bullock, is still the Taguchi Testing guru of the internet marketing world, even if most internet marketers aren’t willing to do the work (or run enough traffic) to make Taguchi work.  (Taguchi is the world’s most advanced statistical model for testing and optimization of complex processes including marketing.)

And I even wrote a book on Taguchi testing (with David’s blessing) with copywriter Bob Bly, way back in the day.

I wish I did more testing today.  Because I know the power.  But more often than not, really high-level direct response marketers are more interested in testing one big package against another — not testing small tweaks that optimize results from a single package.

That said, you’re smart to at least learn the fundamentals.  Because if you’re in direct response marketing long enough, you’ll probably be involved with some tests.  And it pays to know what you’re doing — because…

Testing can actually help you make more money off every marketing campaign you run!

But I’m getting ahead of myself.

Here’s today’s Mailbox Monday question…

Hi Roy,

My question actually correlates with your article about testing.

Quick little background first:

I’ve finished writing the sales letter for my client (potential 5k in my pocket, thanks again Roy) and he is now in the process of making it look pretty to the eye, with testing shortly thereafter.

So, that had me thinking… I’d really enjoy doing some edits.

Maybe some tweaks to the headline, different leads, playing with product’s price — you know, changes with the potential to blast response through the roof.

Then, a couple unknowns popped in my head…

  1. How long (on average) testing tweaks to the copy usually takes (since I’m not in direct control of testing).
  2. How to charge for this (hourly, results-based, fixed fee)… if you even charge at all.
  3. How in-depth the copywriter usually gets with the metrics and overall sales funnel/process — if, for example, my copy failed because the wrong target market was chosen… well, that would be pretty annoying.

I’d love it if you could answer these in an upcoming Mailbox Monday issue.

Have a great day, Roy, and thank you.

Your friend,


(Tjark: pronounced like “Jacques” in “Jacques Cousteau”)

Okay, so there’s a LOT here, and while I go through the questions I’m going to try to hit on some of the fundamentals of testing…

In fact, I’ll start with some fundamentals of marketing testing, from the perspective of a copywriter or other outside consultant.

The first thing you have to know as an outsider is something I hinted at above.  Most clients are not great at testing, and especially ongoing optimization.

Unless you find a die-hard, dyed-in-the-wool direct marketer, they’ll probably mostly pay lip service to testing.

Sure, when it comes to AdWords or Facebook ads where testing is built into the platform, a lot of folks test.  Partially because it’s so dang easy (almost required), but also for another important reason.

Testing requires conversions.  A lot of them.

Now, conversions can be measured a lot of ways.  The easiest conversion to get is from a view to a click.  So, someone sees your Facebook ad, and they click on it.  That’s a “conversion event,” if you want to define it that way, and it can definitely measured.

And anything that can be measured can be tested and optimized.

But you have to understand something important.  To get to a valid test result, you need something called statistical significance.  While you can never be 100% sure that a difference in response between two ads will continue, you can become 95%, 98%, even 99.9% sure, based on statistical models.

That doesn’t happen after 10 views of two different ads, with one getting 2 conversions, and the other getting 1.  That’s not enough to tell you that the one with two conversions will continue to perform at double the one with only one.

Depending on response rates, you may need 100 or 1000 people to see an ad (or more!), and a percentage of those people to convert, if you want to know the result of the test will hold true in future use.

Which means the earlier it is in the funnel — all the way up to the ad that gets people through to your site — the easier it becomes to run valid tests.

That’s the other reason a lot of copy isn’t tested at a level where it will really make a difference.  Because there’s just not enough eyeballs on the copy — and conversions — for it to make sense.

And importantly, it’s worth knowing that not all clicks (or views, or time on page metrics, or whatever) are created equal.

You could have one ad that gets a lot of clicks, but those clicks never turn into customers.  You could have another that gets less clicks, but the revenue per 1,000 views is much higher, because more clicks become customers.

And even there, you could have one ad that brings a lot of first time customers through the door, but none of them stick around.  Whereas you could have another that brings fewer first transactions in, but they all spend a lot more with you.

You’ll always have limitations in what all you can measure, and the less data you have access to, the less actionable your test results are.

That said, I don’t want to totally discourage you — because even bad testing is usually better than no testing!

So, let’s talk about the challenges of getting clients to test…

First off, if you’re a copywriter or consultant, most often you’re not in charge of your client’s web properties.  You have to work with their in-house tech team to get anything done on the site itself.

If they don’t have the culture and tools in place, it can be more hassle than it’s worth to really spend a ton of time getting them to start testing.

I’ve found more and more that I’m most interested in working with the kind of clients who have a culture that fits with what I want to do — and I really hate pushing folks into new territory because it’s like beating your head against a brick wall.

If you are taking more of an agency role where your client has outsourced their web operation to you, you do have a bit of an advantage here.  If you’re running the site, you can install testing software.  If you’re creating landing pages, you could create two alternatives and send traffic to each.

But for the most part, if you don’t have control, you have to find clients who have the capability or the systems in place, and are willing to take on the task of testing.  You can then direct what gets tested, but they have to be on board in the first place.

The length of time a test takes can strongly depend on traffic…

If the client isn’t driving a lot of traffic to the particular sales page, or there aren’t a lot of conversions, tests can take a very long time.

Contrast that with, say, a Google or Amazon test of a very high traffic page, and they can have statistical significance in minutes.

Usually it’s best practice to have a test running for at least 7 days of a normal week, because response can vary considerably between days.  You don’t want to assume what wins on Sunday will win every day of the week.

But if the client isn’t running much traffic, it could take weeks or even months to get any actionable data out of a test.

When I’ve approached testing with clients, I’ve almost always focused on what will get the most eyeballs.

AdWords or Facebook ads, banners, etc.

If you’re thoughtful in this testing — that is, you test ideas you can use later in the sales process — then the data you get here can be very actionable.

In fact, a friend of mine recently told me he was using these kinds of tests to refine ideas for sales copy, before ever writing the main piece.  By the time an idea is turned into a full promotion, it’s highly-refined and likely to be a big winner.

Continuing to try to answer the questions…

How involved do I get with testing?

Again, this really depends.

It’s pretty infrequent that I get highly involved on a nitty-gritty level with clients testing of my marketing pieces.

I’m always more than happy to create multiple versions.  But when it comes down to actual implementation, it’s usually the client who does that, and then we will often chat numbers when there’s something to talk about.

And for the most part, if you’re working with clients who have web teams or manage their own site (versus outsourcing it to you), that’s about the best you should hope for.

There may be exceptions, but they’re not the rule.

That said, I’m often very involved in the strategy going in, developing an entire campaign or funnel for the client.  Sometimes this will include test panels.  When a client is big on testing, I like to be as accommodative as possible because I know it only increases my chances of getting a bigger winner (complete with bigger royalties).

Which leads us to…

How to charge clients when you’re helping with their testing…

For copywriters like me, writing a couple extra test panels as part of a main project takes very little extra time.

When you’re already in the swing of things, to knock out a couple alternative headlines or additional emails is easy, and fast.

And so if I know the client will use them, I usually include them at no cost.

It’s part of my flat project fee plus royalty structure.  I get paid a flat fee to set aside the time for a project, and a royalty on sales generated when I create a winner.  That royalty is a big enough motivation for me to do these extra little things to knock it out of the park.

The better it works, the more royalties I make.  So I don’t charge a nickle-and-dime fee for it.

That said, there are a couple occasions when I might ask for a fee to help with testing…

First if the client comes back to me after an extended length of time, looking for test versions, I might charge a nominal fee.  This is because I have to bring the project and the messaging back to the front of my mind, and get back into it.  Because of the extra effort involved, I like to have that time investment offset with fair compensation.

Second if I were to make testing an important part of my services, or make a specific offer built around it (setting up tests in a testing platform, managing tests and reporting on results, etc.) I’d come up with a reasonable hourly, project-based, or retainer fee to justify this.

This has been a pretty darn technical, nuanced, and not totally positive…

The lesson I learned about testing, back when I wrote the book on Taguchi testing, is that it requires work.

And unfortunately, not a lot of people like work.

So it can be quite difficult to sell testing, unless you’re essentially selling a solution to do it all for them (and even then, that can be difficult if they don’t already understand the power of testing!).

Even many top direct response companies have held onto the direct mail approach to testing.  Test two radically different packages (usually from two high-paid copywriters) against each other, and run with the winner.  Sure, after that they’d run cover tests to try to keep the winner going, and sometimes create an even bigger success.  But it’s nowhere near the level of granular optimization that new technology creates.

That said, like many things that require hard work, there can be big rewards from testing.

If you really get into it or work with clients who are, the breakthroughs you can create may be life-changing.

Yours for bigger breakthroughs,

Roy Furr

Editor, Breakthrough Marketing Secrets

PS — Want to know what Tjark was referring to when he mentioned the $5k in his pocket thanks to my advice?  It’s because he followed my advice on writing an irresistible offer letter for landing your first client, included in my book The Copywriter’s Guide To Getting Paid.  If you’ll pay shipping, I’ll buy the book for you.  Click here to get your free book.