Why does the Septuagint contain non-Tanakh books?

The deuterocanonical books, treated as part of the Bible by the Orthodox and Catholic churches, are accepted because they appear in the Septuagint. However, they are excluded from the Jewish Bible, the Tanakh (and are therefore also excluded by most (all?) Protestant Christians). Given that the Septuagint was a Jewish publication, why does it contain books which are not part of the Tanakh?


I’ve wondered that for a while, which is why I asked the question at Biblical Hermeneutics Stack Exchange. And there I got an answer: basically, at the time the Septuagint was built, the Jewish canon wasn’t finalised. Here’s an informative, but unsourced, answer from JoanW:

The Greek translation of Jewish scripture (the Septuagint) occurred between the 3rd and 1st centuries BCE. The canon of the Tanakh was finalized hundreds of years later. The Christian canon was debated from the 4th to the 16th centuries. We have a tendency of thinking of the Bible as written in stone, so to speak, but the canon has been the object of scholarly debate and change for millennia. Catholic and Orthodox Bibles are generally based on the Septuagint, whereas Protestant Bibles are based on the Masoretic (the authoritative Hebrew text of the Jewish Bible), which was developed between the 7th and 10th centuries CE.

That’s a nice, straightforward overview. I like it.

Here’s a more detailed and referenced answer from Noah Snyder:

In Judaism the final decision of which writings (Ketuvim, the third part of the Tanakh) were canonical did not happen until at least the end of the 1st century CE. This was after Christianity and Judaism had largely split, and so the two groups made different decisions about which writings were accepted as canonical.

In particular, nascent Rabbinic Judaism made the decision to only include writings which were originally written in Hebrew (possibly with parts in Aramaic), but not books which were originally written in Greek (or thought to have been originally written in Greek). The deutorocanonical books are roughly the books which were popular among the Greek speaking Jewish diaspora in the 1st century, but were excluded from the writings on the basis of their being written in Greek (and possibly also excluded for other reasons). Around the same time nascent Rabbinic Judaism also rejected using the Septuagint in favor of only using the original Hebrew texts.

(A key search term to find more to read on this topic is the “Council of Jamnia,” which is the name of the hypothetical council which made the decision on canonizing the Ketuvim. The hypothesis that there was an actual council is no longer particularly popular, instead people tend to think it was a more gradual process, nonetheless the name is still useful for finding material on the topic.)


This entire blog post, both my own writing and the answers I quoted from others, is under the license CC BY-SA 3.0. Feel free to repost elsewhere.

Lightenings viii

The annals say: when the monks of Clonmacnoise
Were all at prayers inside the oratory
A ship appeared above them in the air.

The anchor dragged along behind so deep
It hooked itself into the altar rails
And then, as the big hull rocked to a standstill,

A crewman shinned and grappled down the rope
And struggled to release it. But in vain.
‘This man can’t bear our life here and will drown,’

The abbot said, ‘unless we help him.’ So
They did, the freed ship sailed, and the man climbed back
Out of the marvellous as he had known it.

© Seamus Heaney

Why did Thorongil warn Ecthelion against the White Wizard?

I’m rereading The Lord of the Rings. Appendix A, “Annals of the Kings and Rulers”, tells us that Aragorn son of Arathorn spent part of his youth in Minas Tirith under the assumed name “Thorongil” serving under Ecthelion, Steward of Gondor.

Thorongil often warned Ecthelion not to put trust in Saruman the White in Isengard, but to welcome rather Gandalf the Grey.

Appendix B, “The Tale of Years”, tells us,

2957-80 Aragorn undertakes his great journeys and errantries. As Thorongil he serves in disguise both Thengel of Rohan and Ecthelion II of Gondor.
10th July 3018 Gandalf imprisoned in Orthanc.
18th September 3018 Gandalf escapes from Orthanc in the early hours.
25th October 3018 Council of Elrond.

Saruman’s treachery was not clear to anyone before the dispute with Gandalf in July 3018. And Aragorn did not learn of it till he and Gandalf met again in Rivendell in October. So why was Aragorn already suspicious of Saruman roughly 40 years earlier?


I asked this question a while ago on Science Fiction & Fantasy Stack Exchange. I got a few good answers. Here’s the one I selected as the best, from Peter Turner:


Not sure where this is in the annals, but it says in the Tolkien Companion by J.E.A Tyler

Saruman made his first deliberate move in this direction (toward imposing his will, which was forbidden of the Istari) in the year 2759 Third Age, when he appeared at the Coronation of King Frealaf of Rohan, successor of the mighty Helm Hammerhand. The Wizard brough with him rich presents, and declared himself the friend of Rohan and gondor, and a little later was able to persuade Steward Beren of Gondor to grant him the Keys of Orthanc, the mighty Tower which, together with its fortress of Isengard, commanded the strategic Gap of Rohan. All thought this was a welcome move.

All, that is, except a weary ranger who would see everything given up by Gondor as a challenge to its power.

And it further says that

all the time the Wizard was secretly searching the Tower of Orthanc for a long-lost treasure of the Dunedain … the Palantír of Orthanc.

Then in 2851 the White Council met to think of ways to stop Sauron from coming back

Saruman, hoping that the Ring would expose its location if Sauron were left unharassed, deliberately overruled a strong recommendation (from Gandalf) … that Dol Guldur be attacked.

By his actions, Gandalf may have suspected that Saruman was up to something, although I don’t think Gandalf even knew of the ring.

So, either through his own understanding of the Palantír through the lore of his people or through his association with Gandalf, Aragorn was more naturally suspicious than Gandalf and I think it makes sense that he’d know something was amiss well before anyone else had reason to suspect.


You can read Peter’s answer and all the others at SF&F SE. This entire blog post, both my own writing and the section I quoted from Peter, is under the license CC BY-SA 3.0. Feel free to repost elsewhere.

XML vs JSON

Tu Chu asked a question on Stack Overflow:

I am developing a game for the iOS (and later for Android) devices which needs to get data from a database on a server. What I have done so far is to use PHP to echo out the data from the database as XML. The program will check often with the server so performance is a big deal here. So, would JSON or XML be better for this task?

Well, which is better? I don’t know. It depends on the specific use case, and we don’t have enough detail to answer that question. And this, indeed, is what I said:

Produce XML output. Check the time taken and the file size.

Produce JSON output. Check the time taken and the file size.

Decide which is best.

What more could I say?

When Alice asked a more general question on Programmers Stack Exchange, I was able to say more.

Alice asked,

How important is it to learn XML when JSON is able to do almost all that I need? Having said that, I use JSON mainly for AJAX requests and obtaining data from various APIs. I am a total newbie to web development and the reason I am asking this is that I want to know whether I should go ahead and buy a book on XML or whether I can just give it a pass.

Well, while XML and JSON do have overlaps in use-cases, they are actually very different languages with very different design goals, so I replied,

XML definitely outshines JSON for markup (which is, after all, hinted at in the name).

I wouldn’t like to see a random XHTML page converted into JSON format. It would be horrible. OpenOffice and the latest editions of Microsoft Office all use compressed XML as their format of choice.

As a general rule: Markup goes in XML; structured data goes in JSON.

That’s when you’re outputting data and have full control yourself over the format. If you’re outputting data according to industry standards, or consuming other people’s data, you may need to use XML even in places where JSON would seem more appropriate. That’s because XML is longer established and has been used in many standards.


License: CC BY-SA 3.0. Feel free to repost elsewhere.

Life: bits to be sorted out

The first feminist blog I read with any regularity was Shapely Prose, which I think was a fat acceptance blog which gradually developed into a more general feminist blog, while retaining a focus on HAES. So, for me, learning about feminism and learning about HAES have sort of been bound up together.

I have unhealthy eating habits and unhealthy sleeping habits: the two are bound together, as in “Good grief I’ve been on the computer for how many hours? And now I’m getting a headache from tiredness and I’ll have to skip dinner again.” Then I get up late and skip breakfast. It’s fairly often that I don’t eat from one lunch to the next. Then I eat out in a café because (a) I didn’t have time to make a packed lunch, and (b) I really need a full meal at lunchtime, because I’m not eating at other times. And eating out all the time means that (a) I’m spending far more money on food than I can really afford, and (b) I’m eating too much meat: I try to reduce my meat intake (mainly for environmental reasons; partly for health reasons), and on the rare occasions when I cook for myself I usually cook vegetarian, but when I eat out the meat dishes tempt me. And eating out in the evenings means either posh expensive places or fast food, and vegetarian fast food is rarely appealing to me.

So, I do have unhealthy eating habits I need to get on top of, mainly, probably, by getting better at scheduling my time. Get off the computer in time to cook something, and then get to bed at a reasonable hour and get out of bed at a reasonable hour. This would be good for me.

I’m not entirely sure how to go about this, mind you. Or about the extent to which it interacts with HAES. I’m reasonably happy with my overall body shape.


This post started as a comment on Ana Mardoll’s Ramblings, but as I wrote it it gradually became more and more off topic, so I’m posting it here instead.

How can I send 100,000 e-mails weekly?

Don’t.

That really is the simplest answer.


On Stack Overflow, xRobot asked for guidance on setting up a system which would send 100,000 e-mails every week to a variety of addresses. This is, actually, quite tricky, as was demonstrated in Piskvor‘s rather awesome answer. Here it is:


Short answer: While it’s technically possible to send 100k e-mails each week yourself, the simplest, easiest and cheapest solution is to outsource this to one of the companies that specialize in it (I did say “cheapest”: there’s no limit to the amount of development time (and therefore money) that you can sink into this when trying to DIY).

Long answer: If you decide that you absolutely want to do this yourself, prepare for a world of hurt (after all, this is e-mail/e-fail we’re talking about). You’ll need:

  • e-mail content that is not spam (otherwise you’ll run into additional major roadblocks on every step, even legal repercussions);
  • in addition, your content should be easy to distinguish from spam — that may be a bit hard to do in some cases (I heard that a certain pharmaceutical company had to all but abandon e-mail, as their brand names are quite common in spam mailings);
  • a configurable SMTP server of your own — one which won’t buckle when you dump 100k e-mails onto it (your ISP’s upstream server won’t be sufficient here and you’ll make the ISP violently unhappy; we used two dedicated boxes);
  • some mail wrapper (e.g. PhpMailer if PHP’s your poison of choice; using PHP’s mail() is horrible enough by itself);
  • your own sender function to run in a loop, create the mails and pass them to the wrapper (note that you may run into PHP’s memory limits if your app has a memory leak; you may need to recycle the sending process periodically, or even better, decouple the “creating e-mails” and “sending e-mails” altogether).

Surprisingly, that was the easy part. The hard part is actually sending it:

  • Some servers will ban you when you send too many mails close together, so you need to shuffle and watch your queue (e.g. send one mail to joe@example.com, then three to other domains, only then another to other_address@example.com).
  • You need to have correct PTR, SPF, DKIM records.
  • You need to handle remote server timeouts, misconfigured DNS records, and other network pleasantries.
  • You need to handle invalid e-mails (and no, regex is the wrong tool for that).
  • You need to handle unsubscriptions (many legitimate newsletters have been reclassified as spam due to many frustrated users who couldn’t unsubscribe in one step and instead chose to “mark as spam” — the spam filters do learn, especially with large e-mail providers).
  • You need to handle bounces and rejects (“no such mailbox ojhn@example.com”; “mailbox john@example.com full”).
  • You need to handle blacklisting and removal from blacklists. (Sure, you’re not sending spam. Some recipients won’t be so sure — with such large list, it will happen sometimes, no matter what precautions you take. Some people (e.g., your not-so-scrupulous competitors) might even go as far to falsely report your mailings as spam — it does happen. On average, it takes weeks to get yourself removed from a blacklist.)

And to top it off, you’ll have to manage the legal part of it (various federal, state, and local laws; and even different tangles of laws once you send outside the U.S. (note: you have no way of finding out whether joe@example.com lives in Southwest Elbonia, the country with world’s most draconian antispam laws)).

I’m pretty sure I missed a few heads of this hydra — are you still sure you want to do this yourself? If so, there’ll be another wave, this time merely the annoying problems inherent in sending an e-mail. (You see, SMTP is a store-and-forward protocol, which means that your e-mail will be shuffled across many SMTP servers around the Internet, in the hope that the next one is a bit closer to the final recipient. Basically, the e-mail is sent to an SMTP server, which puts it into its forward queue; when time comes, it will forward it further to a different SMTP server, until it reaches the SMTP server for the given domain. This forward could happen immediately, or in a few minutes, or hours, or days, or never.) Thus, you’ll see the following issues — most of which could happen en route as well as at the destination:

  • The remote SMTP servers don’t want to talk to your SMTP server.
  • Your mails are getting marked as spam (<blink> is not your friend here, nor is <font color=...>).
  • Your mails are delivered days, even weeks late (contrary to popular opinion, SMTP is designed to make a best effort to deliver the message sometime in the future — not to deliver it now).
  • Your mails are not delivered at all (already sent from e-mail server on hop #4, not sent yet from server on hop #5, the server that currently holds the message crashes, data is lost).
  • Your mails are mangled by some poorly designed server en route (this one is somewhat solvable with base64 encoding, but then the size goes up and the e-mail looks more suspicious).
  • Your mails are delivered and the recipients seem not to want them (“I’m sure I didn’t sign up for this, I remember exactly what I did a year ago” (of course you do, sir)).
  • There are problems with users with various versions of Microsoft Outlook and its unique handling of Internet mail.
  • You hit wizard’s apprentice mode (a self-reinforcing positive feedback loop — in other words, automated e-mails as replies to automated e-mails as replies to…; you really don’t want to be the one to set this off, as you’d anger half the internet at yourself).

And it’ll be your job to troubleshoot and solve this (hint: you can’t, mostly). The people who run a legit mass-mailing businesses know that in the end you can’t solve it, and that they can’t solve it either — and they have the reasons well researched, documented and outlined (maybe even as a PowerPoint presentation — complete with sounds and cool transitions — that your bosses can understand), as they’ve had to explain this a million times before. Plus, for the problems that are actually solvable, they know very well how to solve them.

If, after all this, you are not discouraged and still want to do this, go right ahead: it’s even possible that you’ll find a better way to do this. Just know that the road ahead won’t be easy — sending e-mail is trivial, getting it delivered is hard.


I’ve rewritten that slightly to tweak the grammar and to avoid a couple of unnecessary and potentially triggering metaphors. As good as it is, it’s not the last word on the subject. Here’s more advice, from splattne, on how not to be marked as a spammer:


Be sure that your e-mails don’t look like typical spam e-mails: don’t insert only a large image; check that the character-set is set correctly; don’t insert “IP-address only” links. Write your communication as you would write a normal e-mail. Make it really easy to unsubscribe or opt-out. Otherwise, your users will unsubscribe by pressing the “spam” button, and that will affect your reputation.

On the technical side: if you can choose your SMTP server, be sure it is a “clean” SMTP server. IP addresses of spamming SMTP servers are often blacklisted by other providers. If you don’t know your SMTP servers in advance, it’s a good practice to provide configuration options in your application for controlling batch sizes and delay between batches. Some mail servers don’t accept large sending batches or continuous activity.

Use e-mail authentication methods, such as SPF, and Domain Keys to prove that your emails and your domain name belong together. The nice side-effect is you help in preventing that your email domain is spoofed. Also check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.

Make sure that the reply-to address of your emails are a valid, existing addresses. Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Doe" <john.doe@example.com> ) and monitor your abuse accounts, such as abuse@example.com and postmaster@example.com.


Of course, on the other end, it’s also important to protect against spam coming in.


The copyright for the two essays quoted above rests with their original authors. They were originally published on Stack Overflow and Super User, respectively. Both of those essays, and this entire blog post, are under the license CC BY-SA 3.0. Feel free to repost elsewhere.

Am I small-minded?

This thought occurred to me just last night:

There are some opinions I understand and agree with. They are based on arguments and presuppositions which make sense to me, and which seem to me reasonable and well supported.

There are some opinions I disagree with. The arguments in their favour seem to me lacking in some way, perhaps by being based on presuppositions which I do not share, or perhaps due to a failure in logical reasoning from those presuppositions.

There are some opinions I disagree with completely. The arguments in their favour are non-existent, or are based on presuppositions so completely alien to my mind that I simply cannot make sense of them, or follow a chain of logical reasoning which I cannot grasp.

It occurred to me last night that the middle set, the arguments which seem lacking, but not fundamentally unreasonable or utterly incomprehensible, is a quite likely to be a reasonably good proxy for how open-minded we are. (Compare Aristotle’s dictum that it is the mark of an educated mind to be able to entertain an idea without accepting it. What I’m saying is not quite the same thing, but it is a related concept.) It also occurred to me that, for me, the middle set of ideas is, actually, rather small. I find it quite hard to get my head around conservative politics; I find it very difficult to understand or communicate with people who are uninterested in scientific evidence for or against the types of medicine they advocate; and I often find religious concepts difficult to grasp (which last is especially odd, given that I was raised religious).

So, how worried should I be about that?


License: CC BY-SA 3.0. Feel free to repost elsewhere.

Do Jehovah’s Witnesses have their own version of the Bible?

Jehovah’s Witnesses use the standard 66-book Protestant Bible, but usually use their own translation thereof (they do reference other translations from time to time, but generally use The New World Translation). It’s fair to say that the NWT is quite, let’s say, distinctive in places, and has received a fair amount of criticism. The Witnesses do not in any way claim that the translation of the NWT was inspired by God, and are happy to argue their doctrines from other translations if you ask them to. (Indeed, they did so for many years before the release of the NWT, and continue to do so in languages which do not yet have a version of the NWT.)

Here are some of the distinctive features of the New World Translation.


In the specific case of John 1:1, which is always brought up in discussions of the translation philosophy of the NWT, it’s probably fair to say that the Greek is a little ambiguous, and the NWT rendering is defensible. They do, of course, provide a footnote and an appendix article on the subject in The New World Translation — with References.


The terminology is slightly different: what is commonly called the “Old Testament” the Witnesses (and the NWT) call the “Hebrew-Aramaic Scriptures”; what is commonly called the “New Testament” they call the “Christian Greek Scriptures” (the word Christian is intended to prevent any possible confusion with the Septuagint, a Greek translation of the Hebrew Scriptures). This is, arguably, more neutral terminology than the usual. I like it.


The Greek text of the New World Translation is the Westcott and Hort master text, not the Textus Receptus used by the King James Version. There’s a certain amount of dispute in Bible translation circles about which text is better. (The NWT is far from the only translation based on the Westcott and Hort text.) Textus Receptus is based on the majority of ancient manuscripts found. Westcott and Hort is based on fewer, but older, manuscripts. The argument in favour of W&H is simply that older manuscripts are probably better. The argument in favour of TR is that the Bible was copied very carefully and the few old texts which happened to survive merely because they were in Egypt, which has a better climate for this kind of thing, were probably inferior copies. Some also claim that if W&H was the better text, God wouldn’t have allowed it to be lost for so many thousands of years. All of this debate (and yes, I have read a book on the subject, firmly in favour of the Textus Receptus and the related Majority Text) is rendered somewhat moot by the fact that most theologians say that few of the differences between the various Greek texts are theologically meaningful. Footnotes reference other texts and ancient translations (including the Vulgate and Syriac translations) in places, but in general the translation is based on W&H.

(The Hebrew text is far more standard. The NWT uses the Masoretic Text, just like almost everyone else. The footnotes of NWT Ref occasionally reference the Septuagint, the Dead Sea Scrolls, and various Syriac translations, but in general they use the Masoretic.)


One of the distinctive features of the NWT is the use of the name Jehovah. The Tetragrammaton (four-letter name of God) appears multiple times in the Hebrew Scriptures. Many translations render this as LORD, following the Jewish practice of not pronouncing the Divine Name (though the Jews do write the name in their scriptures). The Jerusalem Bible renders the name as Yahweh, which is a scholarly “best guess” at the original pronunciation. The Witnesses use Jehovah, which is almost certainly not the original pronunciation, but is the traditional rendering in English, found in both religious and secular books for many many years. Certainly including some form of the name is more accurate than bowdlerising it.


One of the even more distinctive, and certainly less defensible, features of the NWT is that they also use the name Jehovah in the Greek Scriptures, although it is not found there in any extant manuscript. When the Greek text quotes the Septuagint, they reinsert the name (yes, reinsert, as they maintain that it was there originally). Certainly there do exist editions of the Septuagint which contain the untranslated and untransliterated tetragrammaton, and others which render the divine name as Pipi, suggesting that they were copied from an earlier version which contained the original tetragrammaton, the Hebrew letters of which look a little like the Greek letters for Pipi. (I now feel the need for a fantasy novel in which God is called Pipi. It’s a wonderful name.)

However, NWT includes the divine name in other places too. Sometimes support comes from the existance of the tetragrammaton in Hebrew translations of the Christian Greek Scriptures (some of those Hebrew translations used for support are actually fairly recent, so any support they offer is tenuous at best). The name Jehovah occurs many times in the New World Translation of the Christian Greek Scriptures, each time accomapied by a footnote and an explanation of the rational in NWT Ref.


In general, the footnotes and appendices in NWT Ref are about the mechanics of translation, not theology. They are about tricky linguistic points and textual variants.


Originally published on Christianity Stack Exchange. License: CC BY-SA 3.0. Feel free to repost elsewhere.

Tentative thoughts on Payment Gateways

If you’re selling stuff online, you need a “payment gateway”. That’s something that sits between your website and the bank, so that you can accept money over the Internet. PayPal is the biggest and best known of these. As far as I can tell, different payment gateways work in three different ways. (There may be more, but these are the three I’m aware of. I’m also including one other way of taking money, which is a gateway for payments, but isn’t a “payment gateway” according to the standard definition.) I don’t know of any standard terminology to distinguish these methods, so I’ve invented my own.

So, here’s my breakdown of five types of checkout. (Yes, five.)

1. Orders only

There are, I said, four types of payment gateway (or, three plus a bonus fourth), but five types of checkout. That’s because it’s possible to have something like an online shop where no money actually changes hands. I’ve built a site like this. It displays products, and has nice little “add to cart” buttons, so you can build up an order. You can then review your order and submit it. Then you get an order ID. The owner of the site will then contact you separately to arrange payment and delivery. This method works fine for the site in question (which is for a trade-only wholesale merchant in the fashion business).

This sort of checkout process is by far the easiest to build, as there is no need to interact with* any other system. It’s entirely self-contained.

2. Internal checkout

I call this one “internal” because all the work goes on behind the scenes. You have a form on your website into which the user enters their credit card details. (Watch this: you’re receiving credit card information, so now you’re under a legal obligation to deal with it carefully.) But we don’t store this information, instead, our webserver submits that credit card data to the payment gateway. The response from the gateway indicates whether or not the transaction was successful. (Submitting the information and receiving the response can be a single operation. The exact workflow will depend on the payment gateway concerned. The only one I’m familiar with, Realex Payments, receives the credit card information as a POST request in XML format, and returns the response (also in XML format) immediately.)

Remember, talking to the payment gateway is all happening behind the scenes. The customer has no idea how it works, or which payment gateway you’re using. They enter their credit card details into your website, press submit, and get a response of “payment made” or “declined” or whatever from your site. They have no need or reason to know or care which payment gateway you’re using.

An advantage of the internal checkout method is that you are entirely in control of the user experience. The customer never leaves your site; never sees any logos or branding other than yours. And if you think the checkout process is clunky and difficult to use, you can change it.

Realex Payments, based in Dublin, is one of the largest such operators in the European market. PayPal also supplies this type of payment gateway, but only in the UK, the USA, and Canada.

3. External checkout

For the external checkout, the customer is sent away to an external site to complete the payment process (hence, as you may have guessed, the name). So when the customer has added a few items to their cart, they can click on a button labelled “Pay with X”, proceed to that other site, and pay there. There are a few advantages to this. For a start, customers may be more likely to trust a big site like PayPal or Google with their credit card data. Also, they may already have an account there, and so be able to pay without having to type out their credit card information at all. And, from your perspective, you’re free from having to worry about credit card security: you never see any credit card information at all.

Another thing about the external checkout is that you aren’t limited to just the one of them. There’s nothing to stop you giving the customer options: you can put “Pay with PayPal” and a “Check out with Google Checkout” buttons on the same page, and give the customer a choice. (You can also give the customer the choice of using the internal checkout, of course.)

The bad thing about the external checkout is that they’re a lot trickier to code. The workflow is far more complex. For a start, you have to send the customer away to another site, but with information. There’s no point just sending them away to PayPal. They have to go to PayPal with the information that they’ve come from you, and they’re buying such-and-such which costs so-and-so. The way to do that is to build a form on your site which contains all the necessary information in hidden fields, then use javascript (with a fall back to an image submit button) to send them away to the payment gateway (this is a POST request). (That’s how I did it for PayPal, anyway (and PayPal is the only such external checkout I’ve so far built).)

A while later, the customer will probably arrive back on your site with a similar POST request containing information that they’ve paid. This is all well and good, but how do you know it hasn’t been faked? Also, what if they never do come back to your site? What if they go to PayPal, pay, and then continue on their merry way without visiting your site again? Well, we don’t rely on that.

In our initial submission to PayPal, we send full data about all the products being purchased: ID, title, price, tax, shipping costs, and suchlike, but we also send them a URL and a unique tracking code. When the payment is made, PayPal posts all this data back to that URL (we call it the listener URL, because it sits and listens for PayPal to call it). That’s when we know the payment has gone through. Oh, wait, no we don’t. That could have been faked too. Remember, we aren’t contacting PayPal directly. We haven’t posted any information to PayPal’s servers. We’ve given information (including the unique tracking code and our listener URL) to the customer, and asked them to send it to PayPal. They could be tricking us. The “payment made” response to our listener URL could be a fake too.

So this is the point where we do contact PayPal directly. As soon as we receive the “payment made” call to our listener URL, we send all the information back to PayPal, basically asking “Hey, did this actually come from you?”. PayPal responds immediately, either confirming or denying the call. If PayPal confirms it, we mark the payment as made.

Hang on a second. Someone ordered on our site several thousand euros worth of goods. We created an order, gave it a unique tracking code, and sent them off to PayPal. Then we got word from PayPal that a payment had been made for an order with that tracking code. But the payment was only three euro. What now? And that’s why our listener doesn’t actually mark a payment as made as soon as PayPal confirms it. Instead, it reads all the data from the PayPal response, which includes full details of every item bought, its cost, and all related handling and shipping charges. We then verify that this all matches up with the order we have on record for that unique tracking code. And only then do we mark the payment as made. And we store the order with the PayPal transaction ID (this ID is generated by PayPal, and is unrelated to the code we generated and have used to track the order so far).

When the customer returns to our site, they do so with a POST request from PayPal which includes the transaction ID. This request cannot have been faked, because a faker would have no way of knowing that transaction ID, which was generated by PayPal. So we can be happy that the person landing on our site now is the person who just made the purchase, and we can show a receipt. Everything is hunky-dory.

Except … wait for it. Sometimes, PayPal will return a customer to us before sending the order confirmation to our listener URL. So if a customer lands on our site with a transaction ID we don’t recognize, we can’t simply assume it’s an error. It might be an order which hasn’t come through yet. So we show the page and wait on it a while for an order to come through, checking occasionally (using Ajax) whether an order with that transaction ID has been processed. If it has, we redirect the customer to the receipt for that order. Failing that, we eventually give up.

Just to be slightly more awkward, in PayPal’s sandbox (test environment), there’s no way to force delay the call to the listener URL so you can test this workflow. You just have to code and hope it works. Read that sentence again: Sometimes, PayPal will return a customer to us before sending the order confirmation to our listener URL. Sometimes. There’s no way to force that situation so you can test it.

Also, in case you thought that was too easy, PayPal provides a bunch of other services. It can supply shops which are managed entirely by PayPal: you log into PayPal and create your products and set their prices, and PayPal will give you little HTML snippets to put into your site. And so you can have a shop with no server-side coding on your part at all. This is all very well, but PayPal puts all the documentation for these two completely different situations together into one massive, badly written, repetitive PDF document, and expects you to read it. PayPal’s documentation is easily the worst I’ve ever seen anywhere. It is eye-bleedingly awful. (Realex Payments, by contrast, has very well-written documents. They are to the point, self-contained, and clear.)

The only external checkout I’ve built worked with PayPal. Google Checkout is another provider in this area, and Realex Payments also provide an external checkout service. And, so, no doubt, do many others. As I said earlier, there’s nothing stopping you giving your customer the choice of all these and more.

4. Redirect checkout

The “redirect checkout” is basically an external checkout which pretends to be an internal checkout. The customer enters their credit card details into a form on our site, but the form submits to an external site, which then redirects straight back. The customer, unless they are paying close attention to their browser, does not even realize that they’ve left the site they were on. From a coding perspective, this is essentially the same as an external checkout, but from a user’s perspective, it’s the same as an internal one.

I’ve never actually built a site which used a redirect checkout, and couldn’t name a provider. I’m not making it up, though. It’s something I’m sure I’ve read about somewhere.

5. Telephone payments

Bonus method!

This is the one which doesn’t actually count as a “payment gateway”. The official definition of “payment gateway” is all about credit cards and suchlike, and this method does not require credit cards.

If every product on your site is the same price, and you’re fairly confident that most people will be buying only one product at a time (we’ve done this, for a site which accepted classified advertising), you can accept payment by premium telephone number. First, get a phone number which charges people a fixed price per call, rather than by the second. Then, set up your checkout to create a unique random four-digit number for each order. Then, set up Asterisk to answer the phone, accept input of the four-digit code, and send a signal to to site that the order has been paid.

I did work on a site like this, but I had nothing to do with the Asterisk end of things. It’s a clever program for answering phones and managing phone menu options and phone trees.


* I did not say interface with. You can comment below to thank me for this, if you like.