When TDD can't help you - A case study : Paypal Express Checkout

If you’ve ever been able to work for a period of time where you could use TDD, BDD, or just plain ol’ unit testing, then you’ve probably experienced the joy of having to work the good old fashioned way, what we might call “Brute Force Integration Testing”, otherwise known (when developing web applications) as “Click through the damn thing and see what happens.”

Recently, I’ve had such a joyful experience, and its name is Paypal Express Checkout (henceforth ”EC”).

I’m assuming most everyone knows what Paypal is.  When designing a checkout process, you typically want to have standard credit card processing, and Paypal does offer that.  However, it is a requirement that you also implement EC.  Since that is a standard addition to many a checkout process, this is okay.  Well, sort of.

The Express Checkout flow is pretty basic:

  1. From the shopping cart (as well as from the Payment Information page, more on that later), you click the Checkout with Paypal button….(digression: the button includes the tagline “the safer, easier way to pay.”  Client wants to know why the tagline is there.  I don’t know, it is part of the branding requirements, and I’m a stickler for following agreements.  Client points out that Nike’s store has a button without the tagline.  I point out that Nike probably is able to dictate more favorable terms.  But I digress).
  2. You are redirected to Paypal’s site, where the user logs in, reviews various information, and clicks the Continue button.
  3. You come back to your order review page, and place the order.

Seems simple enough, right?  Right.

The Sandbox

Obviously, you want to test out your implementation of EC before you go live.  Right?  Right.  So, Paypal offers their Sandbox environment.  This is an environment that allows you to make sure things work correctly in an environment that ‘mimics’ how the live environment works.  Sort of.

You set up a developer test account within the sandbox, and it requires you to take various steps that mimic what you would do in the live version.  Sort of.

You login and then you setup some sandbox test accounts.  Specifically, you set up a ‘buyer’ account and a ‘seller’ account.  Because Paypal thinks it is good to mimic the live environment, you have to login to the developer site, then launch the test site and login as the seller, and accept the billing agreement.  This is okay, except if you decide to reset the seller account (  more on this later).

To ‘aid’ in the process, Paypal provides a number of resources.  Specifically, there is an online wizard that generates code for you to create a local test site, as well as sample code to download that shows you how to implement their API.  There’s also a wealth of documentation to read.

In theory, all of this is grand.  You would follow something like the following flow:

  1. Create a local test site.
  2. Run the online wizard to generate the code you need to test out the various steps of the Paypal EC process.
  3. From that code, implement EC in your real site (though still using the sandbox).
  4. Promote to production.

Nothing can possibly go wrong.  Right?  Right.

The 3 Steps of EC

  1. SetExpressCheckout – send initial information to Paypal.  This will create for you a Token, which you will use in the other steps.  It also determines what the user sees when they are redirected to Paypal.
  2. GetExpressCheckout – after the user is redirected from Paypal when clicking the Continue button, this allows you to get any information that was produced when the user was at Paypal.  For instance, the user can set the shipping information, and you want to have this information for your own site.
  3. DoExpressCheckout – this ‘finalizes’ the order (whether it actually finalizes or just authorizes depends on what you tell it to do.  Typically, you don’t capture a payment until the order is shipped, so you typically just authorize the payment.  There are various rules about this, but that is beyond the scope of this post).

Once you begin to implement the three steps, you find a whole bunch of fun stuff.

Online Wizard Generated Code

The skillful, snide reader will come up with a Microsoft joke here, but the code that is generated by the online wizard which is complete, in the sense that it covers all three steps, has a slight flaw.  It won’t compile because of a number of syntax issues.  Not hundreds or anything, but enough to make you notice.  Okay, one syntax issue will make you notice.  This was my first indication that the QA department is not heavily involved with the sandbox environment.  But, that was easy enough to fix, to get it to compile.  And running through the local test site indicated success.  Which is always good.


All calls to Paypal are sent to a quasi-web service.  I say ‘quasi’ because you don’t send the typical web service requests (although there is a SOAP API that lets you go that route), instead, you send key-value pairs to a URL.  For instance, to set the amount of the order being processed, you would send:


Immediately, you will notice that you don’t send ”AMOUNT” but “AMT.”  Why?  Hell, who knows. 

The astute reader will note that, however you implement this within your own code, there is a bunch of magic string magic happening.  I created wrapper classes that allowed for strongly-typed coding within my code, but as you can imagine, I had a few issues where I fat-fingered strings.  This is okay, but one thing about the API is that if you pass in a key (say “AMOUNT” instead of “AMT”) that the Paypal server doesn’t recognize, it doesn’t throw an exception, it just ignores it.  I guess this is okay, but it can cause issues.

Moreover, the API handles versioning with the following:


This turns out to be very important, and very misleading (more in a minute), as certain functionality is only available with certain versions.  As far as things go, I guess this okay.

Passing Order Details and Documentation Fun

One thing that you will naturally want to do is to pass to Paypal the details of the items that are in your shopping cart.  The documentation that is available tells you in great detail how to do this.  Except for a few problems.

For one thing, while there is a lot of documentation that is linked to directly from the Paypal developer site, a lot of important documentation is only findable if you use Google.

Another big thing is that it is not clearly available from any documentation what version is required for certain functionality to be, well, functioning.  If you want to pass Order Details to Paypal, you have to be using Version 53.0 or higher.  If you didn’t search Google and get to the forums, you wouldn’t know this.  Since the online wizard generates code that passes the Version as 51.0, you might theoretically spend a lot of time wondering why Order Details don’t show up in the sandbox.

What makes this even better is that the downloadable sample code, which includes a DLL for you to reference in your .NET solutions, apparently hard-codes the Version to 3.0, so that even if you pass in a different Version, it overrides it.  Which means that if you use the DLL, you simply cannot get Order Details to show up.  Theoretically, this can cause you to spend 10+ hours trying to get this functionality to work in a situation where it simply will never work, unless you dump the DLL and write some code in your actual site that mirrors what the local test site does.

Overriding Shipping Information

When you implement EC, you also have to allow the user to use Paypal from your normal checkout’s payment information page, which will typically come after the user has entered in shipping information.  Naturally, you will want to send this info to Paypal.  This is where the magic string fun comes in.

Paypal’s documentation clearly states how to send in the actual shipping info.  You pass in “SHIPTOSTREET”, “SHIPTOCITY”, etc. etc. etc.  All good.  Except you also have to set the variable “OVERRIDESHIPPING”, or something like that, for Paypal to recognize what you are trying to do.  This is in some of the documentation, I just didn’t see it at first.  Or second.  Or third.  The response that the API sends you won’t tell you if something is missing, it just won’t do what you want it to do.  This is a RTFM issue, but it would be helpful if there was ONE manual to RTF.

The actual testing experience

This is where the disconnect from TDD or any sort of unit testing becomes obvious.

If you want to test how EC works, you have to launch your eCom site and work through the process.  But that’s not all.  In a separate instance or tab of the browser you are testing with, you have to log into the developer Paypal site first.  I’m not entirely sure why, but from the scanning of the forum site I did, it has something to do with how cookies are used. 

This is one of those things where I’m sure there is a solid technical/architectural reason for why it has to work this way, but it is highly annoying.  If your session at the developer Paypal site times out, your testing of your own site will fail.

Moreover, within the sandbox, you can only login with the test accounts you setup.  You have to login with the buyer test account, otherwise nothing works.

As you can imagine, unless you are a Selenium freak of the highest order, you can’t automate any of this.  So, not only do you have to “click through the application and see what happens,” you also have to login to a completely different site ahead of time to get it to work.

In the grand scheme of things, this isn’t necessarily the most horrible thing in the world.  I recently had a conversation with someone complaining that their automated test suite test time went from 2 minutes to 4.  Having been at a contract where it took 10 minutes to build and get to a logon screen, I found this funny.

But once you become addicted to running a single test that takes 10 seconds (at most) to run, having to move to a scenario where you have to click through an application, even if it actually only takes 2 minutes, seems to take an hour.

Magic strings

Another issue I theoretically ran into involved the DoExpressCheckout process, which is what occurs when you get back to your Order Review process and try to process the order.  I was able to get this process to work in the local test site, but I kept getting an error when trying to process it on the ‘real’ local site.

It turns out that I was passing in:


instead of


I’m pretty sure that I saw some documentation that said either would work, but only the latter did.

The ‘hilarious’ thing about it was that the error I was getting only occurred when I was trying to process orders that had multiple items in the cart.  If you only have one item, Paypal doesn’t try to add up the various amounts, so single item orders always worked, even though I was passing in bad info.  Why does it only try to add up amounts when you have multiple items in your cart?  Who the hell knows?

Resetting the Seller Account

When you run through many iterations of testing this lovely thing, you end up with a lot of data and info that you eventually want to get rid of.  So, since the sandbox allows it, you reset the account.  What isn’t documented is that this then breaks the normal Paypal credit card processing, since you have to login as that user in the test Paypal site and re-accept the billing agreement.  And it doesn’t tell you that you need to do this, or how to do it.  You have to click through some non-intuitive links (which you can only find out by visiting the forums) to get this to work.

Final comments

Now that I’ve actually gone through the process, I could implement EC in a day or so.  Of course, given how all of this has gone, I have no confidence that this will actually work in production (still to come) without changes.

When doing TDD or BDD, you are supposed to ask yourself how the API should look.  When dealing with a third-party API, you are stuck with dealing with what you are allowed to do.

It is VERY clear that Paypal doesn’t do much (if any) QA in their sandbox environment.  They don’t update their documentation much with all the relevant details.  Even though I am far from a TDD fanatic, I’m going to go write a program that is TDD-friendly, just so that I can feel better.

posted on Tuesday, July 14, 2009 9:36 PM Print
# re: When TDD can't help you - A case study : Paypal Express Checkout
Chris Missal
7/20/2009 4:47 PM
I hear ya! I keep asking for this over and over on the forums. This post reminded me that I hadn't posted anything there in a long time too.
# re: When TDD can't help you - A case study : Paypal Express Checkout
11/6/2009 10:43 AM
Thanks a lot for your post man!
I spent 5+ hours to show Order Details in sandbox environment.
Lots of googling and finding broken paypal links but no luck.

Finally by keyword "Express Checkout Order Details sample" i googled your post on 2nd page of search results.

Thanks a lot again!
# re: When TDD can't help you - A case study : Paypal Express Checkout
Ed P
11/19/2009 9:03 AM
Just found this post after a good few hours of trying to get order details to display. Changed the version number and problem solved. Thanks!

Post Comment

Title *
Name *
Comment *  
Please add 3 and 3 and type the answer here: