Undefined Behaviour Killed My Cat

schroedinger=/dev/null             # We'll need this guy.
heisenberg=/dev/urandom            # Also needed, for uncertainty principle.
cat $heisenberg > $schroedinger &  # Steal cat from Heisenberg and give it to Schrödinger.
felix=$!                           # Name cat for future references.
exec 2> $schroedinger              # Send all results to Schrödinger.
kill -SIGSTOP $felix               # Catch Felix and put him into a box.
if (($RANDOM & 1))                 # Flip a coin.
then kill $felix                   # Heads: Kill! Kill! Kill!
fi                                 # By now, Felix can be thought of as both alive and dead.
read -sn 1                         # Wait for somebody to open the box.
kill -SIGCONT $felix               # Let him open it.
if ps p $felix > $schroedinger     # Let Schrödinger check on Felix.
then echo The cat is alive.        # Hooray for tails!
else echo The cat is dead.         # At least, now we know.
fi                                 # This concludes the experiment.
kill -SIGKILL $felix               # Felix is no longer required.

From Dennis on Programming Puzzles & Code Golf Stack Exchange. License: CC BY-SA 3.0. Feel free to repost elsewhere.

Why does the Septuagint contain non-Tanakh books?

The deuterocanonical books, treated as part of the Bible by the Orthodox and Catholic churches, are accepted because they appear in the Septuagint. However, they are excluded from the Jewish Bible, the Tanakh (and are therefore also excluded by most (all?) Protestant Christians). Given that the Septuagint was a Jewish publication, why does it contain books which are not part of the Tanakh?

I’ve wondered that for a while, which is why I asked the question at Biblical Hermeneutics Stack Exchange. And there I got an answer: basically, at the time the Septuagint was built, the Jewish canon wasn’t finalised. Here’s an informative, but unsourced, answer from JoanW:

The Greek translation of Jewish scripture (the Septuagint) occurred between the 3rd and 1st centuries BCE. The canon of the Tanakh was finalized hundreds of years later. The Christian canon was debated from the 4th to the 16th centuries. We have a tendency of thinking of the Bible as written in stone, so to speak, but the canon has been the object of scholarly debate and change for millennia. Catholic and Orthodox Bibles are generally based on the Septuagint, whereas Protestant Bibles are based on the Masoretic (the authoritative Hebrew text of the Jewish Bible), which was developed between the 7th and 10th centuries CE.

That’s a nice, straightforward overview. I like it.

Here’s a more detailed and referenced answer from Noah Snyder:

In Judaism the final decision of which writings (Ketuvim, the third part of the Tanakh) were canonical did not happen until at least the end of the 1st century CE. This was after Christianity and Judaism had largely split, and so the two groups made different decisions about which writings were accepted as canonical.

In particular, nascent Rabbinic Judaism made the decision to only include writings which were originally written in Hebrew (possibly with parts in Aramaic), but not books which were originally written in Greek (or thought to have been originally written in Greek). The deutorocanonical books are roughly the books which were popular among the Greek speaking Jewish diaspora in the 1st century, but were excluded from the writings on the basis of their being written in Greek (and possibly also excluded for other reasons). Around the same time nascent Rabbinic Judaism also rejected using the Septuagint in favor of only using the original Hebrew texts.

(A key search term to find more to read on this topic is the “Council of Jamnia,” which is the name of the hypothetical council which made the decision on canonizing the Ketuvim. The hypothesis that there was an actual council is no longer particularly popular, instead people tend to think it was a more gradual process, nonetheless the name is still useful for finding material on the topic.)

This entire blog post, both my own writing and the answers I quoted from others, is under the license CC BY-SA 3.0. Feel free to repost elsewhere.

Lightenings viii

The annals say: when the monks of Clonmacnoise
Were all at prayers inside the oratory
A ship appeared above them in the air.

The anchor dragged along behind so deep
It hooked itself into the altar rails
And then, as the big hull rocked to a standstill,

A crewman shinned and grappled down the rope
And struggled to release it. But in vain.
‘This man can’t bear our life here and will drown,’

The abbot said, ‘unless we help him.’ So
They did, the freed ship sailed, and the man climbed back
Out of the marvellous as he had known it.

© Seamus Heaney

Why did Thorongil warn Ecthelion against the White Wizard?

I’m rereading The Lord of the Rings. Appendix A, “Annals of the Kings and Rulers”, tells us that Aragorn son of Arathorn spent part of his youth in Minas Tirith under the assumed name “Thorongil” serving under Ecthelion, Steward of Gondor.

Thorongil often warned Ecthelion not to put trust in Saruman the White in Isengard, but to welcome rather Gandalf the Grey.

Appendix B, “The Tale of Years”, tells us,

2957-80 Aragorn undertakes his great journeys and errantries. As Thorongil he serves in disguise both Thengel of Rohan and Ecthelion II of Gondor.
10th July 3018 Gandalf imprisoned in Orthanc.
18th September 3018 Gandalf escapes from Orthanc in the early hours.
25th October 3018 Council of Elrond.

Saruman’s treachery was not clear to anyone before the dispute with Gandalf in July 3018. And Aragorn did not learn of it till he and Gandalf met again in Rivendell in October. So why was Aragorn already suspicious of Saruman roughly 40 years earlier?

I asked this question a while ago on Science Fiction & Fantasy Stack Exchange. I got a few good answers. Here’s the one I selected as the best, from Peter Turner:

Not sure where this is in the annals, but it says in the Tolkien Companion by J.E.A Tyler

Saruman made his first deliberate move in this direction (toward imposing his will, which was forbidden of the Istari) in the year 2759 Third Age, when he appeared at the Coronation of King Frealaf of Rohan, successor of the mighty Helm Hammerhand. The Wizard brough with him rich presents, and declared himself the friend of Rohan and gondor, and a little later was able to persuade Steward Beren of Gondor to grant him the Keys of Orthanc, the mighty Tower which, together with its fortress of Isengard, commanded the strategic Gap of Rohan. All thought this was a welcome move.

All, that is, except a weary ranger who would see everything given up by Gondor as a challenge to its power.

And it further says that

all the time the Wizard was secretly searching the Tower of Orthanc for a long-lost treasure of the Dunedain … the Palantír of Orthanc.

Then in 2851 the White Council met to think of ways to stop Sauron from coming back

Saruman, hoping that the Ring would expose its location if Sauron were left unharassed, deliberately overruled a strong recommendation (from Gandalf) … that Dol Guldur be attacked.

By his actions, Gandalf may have suspected that Saruman was up to something, although I don’t think Gandalf even knew of the ring.

So, either through his own understanding of the Palantír through the lore of his people or through his association with Gandalf, Aragorn was more naturally suspicious than Gandalf and I think it makes sense that he’d know something was amiss well before anyone else had reason to suspect.

You can read Peter’s answer and all the others at SF&F SE. This entire blog post, both my own writing and the section I quoted from Peter, is under the license CC BY-SA 3.0. Feel free to repost elsewhere.


Tu Chu asked a question on Stack Overflow:

I am developing a game for the iOS (and later for Android) devices which needs to get data from a database on a server. What I have done so far is to use PHP to echo out the data from the database as XML. The program will check often with the server so performance is a big deal here. So, would JSON or XML be better for this task?

Well, which is better? I don’t know. It depends on the specific use case, and we don’t have enough detail to answer that question. And this, indeed, is what I said:

Produce XML output. Check the time taken and the file size.

Produce JSON output. Check the time taken and the file size.

Decide which is best.

What more could I say?

When Alice asked a more general question on Programmers Stack Exchange, I was able to say more.

Alice asked,

How important is it to learn XML when JSON is able to do almost all that I need? Having said that, I use JSON mainly for AJAX requests and obtaining data from various APIs. I am a total newbie to web development and the reason I am asking this is that I want to know whether I should go ahead and buy a book on XML or whether I can just give it a pass.

Well, while XML and JSON do have overlaps in use-cases, they are actually very different languages with very different design goals, so I replied,

XML definitely outshines JSON for markup (which is, after all, hinted at in the name).

I wouldn’t like to see a random XHTML page converted into JSON format. It would be horrible. OpenOffice and the latest editions of Microsoft Office all use compressed XML as their format of choice.

As a general rule: Markup goes in XML; structured data goes in JSON.

That’s when you’re outputting data and have full control yourself over the format. If you’re outputting data according to industry standards, or consuming other people’s data, you may need to use XML even in places where JSON would seem more appropriate. That’s because XML is longer established and has been used in many standards.

License: CC BY-SA 3.0. Feel free to repost elsewhere.

Life: bits to be sorted out

The first feminist blog I read with any regularity was Shapely Prose, which I think was a fat acceptance blog which gradually developed into a more general feminist blog, while retaining a focus on HAES. So, for me, learning about feminism and learning about HAES have sort of been bound up together.

I have unhealthy eating habits and unhealthy sleeping habits: the two are bound together, as in “Good grief I’ve been on the computer for how many hours? And now I’m getting a headache from tiredness and I’ll have to skip dinner again.” Then I get up late and skip breakfast. It’s fairly often that I don’t eat from one lunch to the next. Then I eat out in a café because (a) I didn’t have time to make a packed lunch, and (b) I really need a full meal at lunchtime, because I’m not eating at other times. And eating out all the time means that (a) I’m spending far more money on food than I can really afford, and (b) I’m eating too much meat: I try to reduce my meat intake (mainly for environmental reasons; partly for health reasons), and on the rare occasions when I cook for myself I usually cook vegetarian, but when I eat out the meat dishes tempt me. And eating out in the evenings means either posh expensive places or fast food, and vegetarian fast food is rarely appealing to me.

So, I do have unhealthy eating habits I need to get on top of, mainly, probably, by getting better at scheduling my time. Get off the computer in time to cook something, and then get to bed at a reasonable hour and get out of bed at a reasonable hour. This would be good for me.

I’m not entirely sure how to go about this, mind you. Or about the extent to which it interacts with HAES. I’m reasonably happy with my overall body shape.

This post started as a comment on Ana Mardoll’s Ramblings, but as I wrote it it gradually became more and more off topic, so I’m posting it here instead.

How can I send 100,000 e-mails weekly?


That really is the simplest answer.

On Stack Overflow, xRobot asked for guidance on setting up a system which would send 100,000 e-mails every week to a variety of addresses. This is, actually, quite tricky, as was demonstrated in Piskvor‘s rather awesome answer. Here it is:

Short answer: While it’s technically possible to send 100k e-mails each week yourself, the simplest, easiest and cheapest solution is to outsource this to one of the companies that specialize in it (I did say “cheapest”: there’s no limit to the amount of development time (and therefore money) that you can sink into this when trying to DIY).

Long answer: If you decide that you absolutely want to do this yourself, prepare for a world of hurt (after all, this is e-mail/e-fail we’re talking about). You’ll need:

  • e-mail content that is not spam (otherwise you’ll run into additional major roadblocks on every step, even legal repercussions);
  • in addition, your content should be easy to distinguish from spam — that may be a bit hard to do in some cases (I heard that a certain pharmaceutical company had to all but abandon e-mail, as their brand names are quite common in spam mailings);
  • a configurable SMTP server of your own — one which won’t buckle when you dump 100k e-mails onto it (your ISP’s upstream server won’t be sufficient here and you’ll make the ISP violently unhappy; we used two dedicated boxes);
  • some mail wrapper (e.g. PhpMailer if PHP’s your poison of choice; using PHP’s mail() is horrible enough by itself);
  • your own sender function to run in a loop, create the mails and pass them to the wrapper (note that you may run into PHP’s memory limits if your app has a memory leak; you may need to recycle the sending process periodically, or even better, decouple the “creating e-mails” and “sending e-mails” altogether).

Surprisingly, that was the easy part. The hard part is actually sending it:

  • Some servers will ban you when you send too many mails close together, so you need to shuffle and watch your queue (e.g. send one mail to joe@example.com, then three to other domains, only then another to other_address@example.com).
  • You need to have correct PTR, SPF, DKIM records.
  • You need to handle remote server timeouts, misconfigured DNS records, and other network pleasantries.
  • You need to handle invalid e-mails (and no, regex is the wrong tool for that).
  • You need to handle unsubscriptions (many legitimate newsletters have been reclassified as spam due to many frustrated users who couldn’t unsubscribe in one step and instead chose to “mark as spam” — the spam filters do learn, especially with large e-mail providers).
  • You need to handle bounces and rejects (“no such mailbox ojhn@example.com”; “mailbox john@example.com full”).
  • You need to handle blacklisting and removal from blacklists. (Sure, you’re not sending spam. Some recipients won’t be so sure — with such large list, it will happen sometimes, no matter what precautions you take. Some people (e.g., your not-so-scrupulous competitors) might even go as far to falsely report your mailings as spam — it does happen. On average, it takes weeks to get yourself removed from a blacklist.)

And to top it off, you’ll have to manage the legal part of it (various federal, state, and local laws; and even different tangles of laws once you send outside the U.S. (note: you have no way of finding out whether joe@example.com lives in Southwest Elbonia, the country with world’s most draconian antispam laws)).

I’m pretty sure I missed a few heads of this hydra — are you still sure you want to do this yourself? If so, there’ll be another wave, this time merely the annoying problems inherent in sending an e-mail. (You see, SMTP is a store-and-forward protocol, which means that your e-mail will be shuffled across many SMTP servers around the Internet, in the hope that the next one is a bit closer to the final recipient. Basically, the e-mail is sent to an SMTP server, which puts it into its forward queue; when time comes, it will forward it further to a different SMTP server, until it reaches the SMTP server for the given domain. This forward could happen immediately, or in a few minutes, or hours, or days, or never.) Thus, you’ll see the following issues — most of which could happen en route as well as at the destination:

  • The remote SMTP servers don’t want to talk to your SMTP server.
  • Your mails are getting marked as spam (<blink> is not your friend here, nor is <font color=...>).
  • Your mails are delivered days, even weeks late (contrary to popular opinion, SMTP is designed to make a best effort to deliver the message sometime in the future — not to deliver it now).
  • Your mails are not delivered at all (already sent from e-mail server on hop #4, not sent yet from server on hop #5, the server that currently holds the message crashes, data is lost).
  • Your mails are mangled by some poorly designed server en route (this one is somewhat solvable with base64 encoding, but then the size goes up and the e-mail looks more suspicious).
  • Your mails are delivered and the recipients seem not to want them (“I’m sure I didn’t sign up for this, I remember exactly what I did a year ago” (of course you do, sir)).
  • There are problems with users with various versions of Microsoft Outlook and its unique handling of Internet mail.
  • You hit wizard’s apprentice mode (a self-reinforcing positive feedback loop — in other words, automated e-mails as replies to automated e-mails as replies to…; you really don’t want to be the one to set this off, as you’d anger half the internet at yourself).

And it’ll be your job to troubleshoot and solve this (hint: you can’t, mostly). The people who run a legit mass-mailing businesses know that in the end you can’t solve it, and that they can’t solve it either — and they have the reasons well researched, documented and outlined (maybe even as a PowerPoint presentation — complete with sounds and cool transitions — that your bosses can understand), as they’ve had to explain this a million times before. Plus, for the problems that are actually solvable, they know very well how to solve them.

If, after all this, you are not discouraged and still want to do this, go right ahead: it’s even possible that you’ll find a better way to do this. Just know that the road ahead won’t be easy — sending e-mail is trivial, getting it delivered is hard.

I’ve rewritten that slightly to tweak the grammar and to avoid a couple of unnecessary and potentially triggering metaphors. As good as it is, it’s not the last word on the subject. Here’s more advice, from splattne, on how not to be marked as a spammer:

Be sure that your e-mails don’t look like typical spam e-mails: don’t insert only a large image; check that the character-set is set correctly; don’t insert “IP-address only” links. Write your communication as you would write a normal e-mail. Make it really easy to unsubscribe or opt-out. Otherwise, your users will unsubscribe by pressing the “spam” button, and that will affect your reputation.

On the technical side: if you can choose your SMTP server, be sure it is a “clean” SMTP server. IP addresses of spamming SMTP servers are often blacklisted by other providers. If you don’t know your SMTP servers in advance, it’s a good practice to provide configuration options in your application for controlling batch sizes and delay between batches. Some mail servers don’t accept large sending batches or continuous activity.

Use e-mail authentication methods, such as SPF, and Domain Keys to prove that your emails and your domain name belong together. The nice side-effect is you help in preventing that your email domain is spoofed. Also check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.

Make sure that the reply-to address of your emails are a valid, existing addresses. Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Doe" <john.doe@example.com> ) and monitor your abuse accounts, such as abuse@example.com and postmaster@example.com.

Of course, on the other end, it’s also important to protect against spam coming in.

The copyright for the two essays quoted above rests with their original authors. They were originally published on Stack Overflow and Super User, respectively. Both of those essays, and this entire blog post, are under the license CC BY-SA 3.0. Feel free to repost elsewhere.

Am I small-minded?

This thought occurred to me just last night:

There are some opinions I understand and agree with. They are based on arguments and presuppositions which make sense to me, and which seem to me reasonable and well supported.

There are some opinions I disagree with. The arguments in their favour seem to me lacking in some way, perhaps by being based on presuppositions which I do not share, or perhaps due to a failure in logical reasoning from those presuppositions.

There are some opinions I disagree with completely. The arguments in their favour are non-existent, or are based on presuppositions so completely alien to my mind that I simply cannot make sense of them, or follow a chain of logical reasoning which I cannot grasp.

It occurred to me last night that the middle set, the arguments which seem lacking, but not fundamentally unreasonable or utterly incomprehensible, is a quite likely to be a reasonably good proxy for how open-minded we are. (Compare Aristotle’s dictum that it is the mark of an educated mind to be able to entertain an idea without accepting it. What I’m saying is not quite the same thing, but it is a related concept.) It also occurred to me that, for me, the middle set of ideas is, actually, rather small. I find it quite hard to get my head around conservative politics; I find it very difficult to understand or communicate with people who are uninterested in scientific evidence for or against the types of medicine they advocate; and I often find religious concepts difficult to grasp (which last is especially odd, given that I was raised religious).

So, how worried should I be about that?

License: CC BY-SA 3.0. Feel free to repost elsewhere.

Do Jehovah’s Witnesses have their own version of the Bible?

Jehovah’s Witnesses use the standard 66-book Protestant Bible, but usually use their own translation thereof (they do reference other translations from time to time, but generally use The New World Translation). It’s fair to say that the NWT is quite, let’s say, distinctive in places, and has received a fair amount of criticism. The Witnesses do not in any way claim that the translation of the NWT was inspired by God, and are happy to argue their doctrines from other translations if you ask them to. (Indeed, they did so for many years before the release of the NWT, and continue to do so in languages which do not yet have a version of the NWT.)

Here are some of the distinctive features of the New World Translation.

In the specific case of John 1:1, which is always brought up in discussions of the translation philosophy of the NWT, it’s probably fair to say that the Greek is a little ambiguous, and the NWT rendering is defensible. They do, of course, provide a footnote and an appendix article on the subject in The New World Translation — with References.

The terminology is slightly different: what is commonly called the “Old Testament” the Witnesses (and the NWT) call the “Hebrew-Aramaic Scriptures”; what is commonly called the “New Testament” they call the “Christian Greek Scriptures” (the word Christian is intended to prevent any possible confusion with the Septuagint, a Greek translation of the Hebrew Scriptures). This is, arguably, more neutral terminology than the usual. I like it.

The Greek text of the New World Translation is the Westcott and Hort master text, not the Textus Receptus used by the King James Version. There’s a certain amount of dispute in Bible translation circles about which text is better. (The NWT is far from the only translation based on the Westcott and Hort text.) Textus Receptus is based on the majority of ancient manuscripts found. Westcott and Hort is based on fewer, but older, manuscripts. The argument in favour of W&H is simply that older manuscripts are probably better. The argument in favour of TR is that the Bible was copied very carefully and the few old texts which happened to survive merely because they were in Egypt, which has a better climate for this kind of thing, were probably inferior copies. Some also claim that if W&H was the better text, God wouldn’t have allowed it to be lost for so many thousands of years. All of this debate (and yes, I have read a book on the subject, firmly in favour of the Textus Receptus and the related Majority Text) is rendered somewhat moot by the fact that most theologians say that few of the differences between the various Greek texts are theologically meaningful. Footnotes reference other texts and ancient translations (including the Vulgate and Syriac translations) in places, but in general the translation is based on W&H.

(The Hebrew text is far more standard. The NWT uses the Masoretic Text, just like almost everyone else. The footnotes of NWT Ref occasionally reference the Septuagint, the Dead Sea Scrolls, and various Syriac translations, but in general they use the Masoretic.)

One of the distinctive features of the NWT is the use of the name Jehovah. The Tetragrammaton (four-letter name of God) appears multiple times in the Hebrew Scriptures. Many translations render this as LORD, following the Jewish practice of not pronouncing the Divine Name (though the Jews do write the name in their scriptures). The Jerusalem Bible renders the name as Yahweh, which is a scholarly “best guess” at the original pronunciation. The Witnesses use Jehovah, which is almost certainly not the original pronunciation, but is the traditional rendering in English, found in both religious and secular books for many many years. Certainly including some form of the name is more accurate than bowdlerising it.

One of the even more distinctive, and certainly less defensible, features of the NWT is that they also use the name Jehovah in the Greek Scriptures, although it is not found there in any extant manuscript. When the Greek text quotes the Septuagint, they reinsert the name (yes, reinsert, as they maintain that it was there originally). Certainly there do exist editions of the Septuagint which contain the untranslated and untransliterated tetragrammaton, and others which render the divine name as Pipi, suggesting that they were copied from an earlier version which contained the original tetragrammaton, the Hebrew letters of which look a little like the Greek letters for Pipi. (I now feel the need for a fantasy novel in which God is called Pipi. It’s a wonderful name.)

However, NWT includes the divine name in other places too. Sometimes support comes from the existance of the tetragrammaton in Hebrew translations of the Christian Greek Scriptures (some of those Hebrew translations used for support are actually fairly recent, so any support they offer is tenuous at best). The name Jehovah occurs many times in the New World Translation of the Christian Greek Scriptures, each time accomapied by a footnote and an explanation of the rational in NWT Ref.

In general, the footnotes and appendices in NWT Ref are about the mechanics of translation, not theology. They are about tricky linguistic points and textual variants.

Originally published on Christianity Stack Exchange. License: CC BY-SA 3.0. Feel free to repost elsewhere.