The way to test this is to have your tours go equally to your billing partners, in real time. You simply cannot switch out one for the other because there are too many variables.

Write a simple PHP script that effectively rotates between your billers -- 1, 2, 3, 1, 2, 3, and so on. For any time period of the day each biller should get the same number of hits.

Even this requires sufficient volume from your join page (I'd say at least 5-10 an hour), and it won't be a true scientific test if you have cascading. You'll have to go through a day or two without cascading.

At the end of the test you'll know once and for all which biller is providing the better conversions. End of story.

Well.....end of Chapter 7, so to speak. :) There's more:

We have this script at MRPW, and have done tests like you described. Our better-performing sites can get dozens of hits an hour. Problem is: there are still many factors beyond a test's capabilities, factors like merchant accounts, the various and same banks processors may use and share, stock markets, currency exchanges, Visa being picky, nosy, upcoming elections therefore political hanky-panky and pressures...on and fucking on.

BTW: 9/26 is about when I noticed the disturbing "yo-yo" affect again (hadn't reared its' ugly head since mid-spring or so): One day UP, next day DOWN, next day UP, next day DOWN...and so on.

As my good friend Vjo says: "Ride the wave, baby, ride the wave..." or maybe that was Cunningstunt? No, too optimistic for him...Teencat? No, too gramatically correct for that (which also rules out FatFoo)....Jesus, too much time on GFY. Ciao.

cant complain

Same exact date I mentioned in my post, 9/26.

Who cares. These have nothing to do with whether CCBill is to blame, which is what this thread has become. The test is merely to show if one processor really outperforms another, not why.

If you divvy up the joins you send to your billers in an equal but random fashion, and do it in real time so the comparison is timely and current (not testing day to day, or week to week) you won't know the root cause of the slowdowns, but you can say for certain processor X outperforms processor Y.

This is how any real business would conduct these tests. They draw off customers at random, but all at the same time and place. Y'all are complicating what is really a very simple process.

Guess you missed the part where I said we did test this. I never said I didn't test the same time, day of week, etc. My point was, these other factors can have an affect on any 3rd party cc processer.

In other words, if testing would simply answer the issue don't you think a "real business" like Twistys (forget MRPW) would have their answer and not have to start threads publicly wondering WTF? Guess testing doesn't answer it then, does it?

I guess you missed the part where I said the test is to flatly determine if a processor is to blame. You said you ran the tests but since you still complain about strange sales days the conclusion has to be it's not the fault of the specific processor. It's one of these other things, and you are right, there are many things that influence a day's sales.

A/B tests are common in research and business. But you must pull from the same test group. You don't test one processor one day, and another the next, because those are different test groups. You don't switch processors when things start to look strange. Every time I read one of these threads this is what people are doing. Shap didn't say he's run A/B tests. He said he flipped the cascade after noticing a period of poor sales. That's inconclusive.

Or, just on the fundamental analysis side, I note Kickass still uses CCBill at the top of their cascade. They're pretty big, right? Either they accept these mysterious sales days as part of business, or they don't have them, or CCBill has a secret pact with them to not screw them over, like they supposedly do with everyone else.

Don't get me wrong - I'm not saying we should blindly accept results from third party partners and never question things. Questioning is good. But there are proven, established methodologies that can resolve these questions. If you've done A/B tests, it would be nice to know the (general) results. Include the length of the test, the size of the test groups, and what you discovered.

I mean, it's not like this information would help a competitor, but if whatever the outcome of your tests, it would provide some real facts among a lot of conjecture and outright fearmongering.

Tests:

A/B splits with CCBill and Epoch, over 5 days, 24 hours a day, 5 days straight.
Following week: 4 days of A/B testing 24 hours, 4 days straight.

Same results both times: When CCBill sales were down, Epoch sales were up, and vice versa.

We've also done speed tests on join form load times:
CCBill (average): 2.5 seconds
Epoch: 3.1 seconds (add the re-direct time to that, making for a total of 3.9 seconds; switch the cascade and similar time frames result with both processors)

Now i won't break down sales vs. percentages vs. etc etc etc. But the point is simple: We've done tests. This issue is not relegated to this week, either, so we've been "investigating" these odd patterns since around April-May.

To answer the other part: WHY some companies still use CCBill despite the above issues? Simple; CCBill IS reliable, they always pay on time, and, in my case, my Affiliate Program is a CCBill affiliate program. Finally, while getting my own merch account, or using NATS, or simply controlling the billing process more would be optimal, at the moment I'm more-or-less a single person, handling everything. So the idea of writing checks weekly to hundreds of affiliates doesn't thrill me. LOL Talk to me next year (2011) however, and if these "odd patterns" continue, and our tests show us that we're leaving thousands on the table by NOT fully controlling our own billing, then the situation shall change, trust me.

Your points are all good ones, BTW. It's just when one assumes we haven't tried someone's great suggestions, or when someone implies we're too stoopid to try testing I go all keyboard. LOL

many of the people posting and complaining in this thread would be better off if they spent more time worry about traffic.

IMO ofcourse

Just to followup, my sales came back today. After days of subpar perfomance I suddenly had a regular days worth of sales by lunch. Oh well, this week will end up only 10% lower than last week so I can't complain.

Update:
Came back with a roar this weekend! Made up for the rest of the week. Who knows? :thumbsup
Piper