Hypothesis testing proves ESP is real

Or maybe it would be more accurate to say that ESP proves frequentist statistics isn’t real. In what has got to be one of the best object lessons in why hypothesis testing (the same statistical method usually used by the medical research industry to produce the Scare of the Week) is prone to generate false results, a well-respected psychology journal is set to publish a paper describing a statistically significant finding that ESP works.

The real news here, of course, is not that ESP has been proven real, but that using statistics to try to understand the world is a breeding ground for junk science. If I had to guess, I’d say the author (who is a well-respected academic who has never published any previous work on ESP) has had a case of late-career integrity and has decided to play a wonderful joke on the all-too-deserving field of psychology by doing a few experiments until obtaining statistically meaningful results that defy everything we know about the universe. As I’ve pointed out before, one of the many problems with hypothesis testing is that you don’t have to try all that hard to prove anything, even when done “correctly,” given that nobody really keeps track of negative results.

Finally, a good Subversion client for Mac OS X

If you don’t have a Mac, or don’t know what SVN is, please accept my apologies for this very directed post. To the one guy remaining, rejoice:

For the longest time, there has been no good SVN interface available on the Mac. Windows folks had TortoiseSVN, and Linux folks wouldn’t be caught dead using anything other than command line tools (or, git, for that matter). So, everybody was happy but us Mac folks.

A program called “Versions” has been available for a while, but it, sadly, epitomizes the style over substance sin that is so prevalent on the Mac. It’s got a beautiful interface, but it’s an interface to very little. Namely, it doesn’t support merging or branching, which is pretty much the most important reason for using a versioning system like SVN. If you’re not branching and merging, you might as well just use a good backup system, because that’s pretty much all you’re using SVN for at that point.

So, I was very excited to find “Cornerstone,” which was recently upgraded to support the slickest SVN interface I’ve seen on any platform. It’s as pretty as “Versions” and as powerful (if not moreso) than TortoiseSVN. It’s merge facility is the best approach I’ve seen, for example. It’s intuitive, and as you adjust the settings it automatically performs a trial merge and gives you the results in real time. Awesome.

They have a two-week trial, which is more than enough to get a feel for the product, it’s so simple and well-executed.

(By the way, they aren’t giving me anything for this. I wish they were, but I don’t have that kind of juice.)

Getting the most data speed out of your cell phone

You may have noticed there have been very few posts here. There’s a reason for that. The first and foremost is that sending my rants in to the void has not been as personally cathartic as I’d hoped. My other goal for the blog, which actually has been somewhat successful, was to simply provide a vehicle for putting information out on the web that I thought might be useful for people, and that I couldn’t find elsewhere. Based on the traffic stats, those posts have actually been worthwhile, and my only reason for not doing more of this kind of post has been that I’ve been too busy playing with my son, finishing up my projects at MIT, and trying to get a job (in that order).

So, going forward, I’m just going to focus on the second category of posts (though I reserve the right to devolve to the first occasionally). This blog was getting too negative, anyway. In that spirit, here’s a particularly useful trick I just figured out while sitting in a coffee shop working remotely.

I recently gave up my nice window office since I was feeling guilty about taking up a nice spot but only working part time. So, I’ve been doing a lot of work remotely, usually from a coffee shop given that working at home just isn’t very productive when there’s an adorable toddler running around begging to be hugged. So, I splurged and decided to start paying the extra $20 a month to use my phone as an internet connection for my computer. This is becoming a pretty common thing, and Sprint even offers phones that will create a WiFi network on the fly (I use Bluetooth with my iPhone). I expect this will become even more common once the iPhone hits Verizon, as Apple will reportedly allow this version of their phone to create WiFi hotspots, too.

I would typically just leave my phone laying flat on the table next to my laptop. However, giving it a minute of thought, this is actually pretty dumb, for two reasons. First, having the phone so close to the laptop is probably not smart, as computers are notorious spewers of electromagnetic interference at pretty much every frequency imaginable. In theory, they should be shielded, but nothing is perfect and between the memory data rates and the processor clock speeds, a computer pretty much has the cell phone spectrum covered directly, if not with overtones. So, keep the cell phone away form the computer at least a foot or so.

Most importantly, however, leaving the cell phone flat on a table is a bad idea because it puts the antenna horizontal, whereas cell phone signals are polarized vertically. (What this means, if you’re not a fan of electromagnetics, is that the electrons in the cell phone tower antenna are being shaken up and down, not side-to-side. Radio waves are really just a way of keeping track of how electrons interact with each other. Without anything interfering, the electrons in your cell phone’s antenna will be wiggled in the same orientation and frequency as those in the cell tower antenna. However, antennas are designed for their electrons to be wiggled in a certain direction (it’s almost always along the long axis of the antenna) and a cell phone’s antenna is oriented with the assumption that the user is holding it upright against their ear.) Once I realized this, I put my phone up against a nearby wall so that it was standing straight up and down (as if somebody were holding it) and my data rates nearly doubled.

So, if you’re using your cell phone as an internet connection, keep it a bit away from the computer and prop it up so it’s vertical. Keeping it vertical in your pocket probably isn’t a great idea, since your body is pretty good at blocking radio. If you find this helps, please let me know in the comments. Right now my experience alone isn’t very statistically significant, to say the least.

Why you should stick with AT&T if you have an iPhone

It was just announced that Apple will finally port the iPhone over to Verizon’s network early next year. The conventional wisdom being that AT&T is an incompetent foil to Apple’s engineering genius, the only thing holding back the iPhone from true greatness, virtually everybody I know with an iPhone (and many waiting) say that they can’t wait until they can get a Verizon iPhone.

Let me pour a little rain on this parade. I’m tempted to say nothing (and let’s be honest, writing on this blog is pretty close to doing just that) because I’d love to have everybody run away to Verizon to clog up their network while those of us staying with AT&T enjoy the highest speeds we’ve ever seen. However, I suspect anybody moving from AT&T to Verizon will be sorely disappointed, for a few reasons.

First, it’s now widely acknowledged that the reception problems with the iPhone 3G, and to some extent the iPhone 4, are entirely the fault of Apple. It’s pretty clear from comparisons with AT&T network performance on other brands of phones versus the iPhone that Apple had a lot of learning to do about writing baseband wireless software. Apparently making a good cell phone is more than just sourcing a few million chips from Infineon and then treating the rest of the phone like a small laptop. Apple was way behind on the RF engineering needed to make a reliable cell phone. Even now, this is evident in the poor (albeit improved) performance of the iPhone 4, which can’t seem to figure out how to keep connected to a good signal and requires frequent cycling of the wireless chip to maintain a good connection. So, unless you live in an area just not well-served by AT&T, you will likely find slower speeds on Verizon. While Verizon does cover more physical space with their network, AT&T’s network is provably faster where it actually exists.

The above brings me to the second point: if Apple had a bit of a learning curve to figuring out how to write firmware for a GSM phone, it stands to reason they might have a few initial hiccups with a CDMA phone. Verizon’s network operates on a fundamentally different standard than AT&T’s, and Apple will be using wireless chips from a different company (Qualcomm) in their Verizon-compatible phones going forward. Given Apple’s propensity to punish the hell out of early adopters, and having paid my dues in that regard, I have no intention of seeing how they manage to screw up connectivity to Verizon’s network.

Finally, if the problem really is, to some extent, AT&T being overloaded by iPhone users, it would seem to me that the last thing you want to do is be part of the stampede over to Verizon. Just as things are finally speeding up for us sticking with AT&T, the poor existing Verizon folks will be waiting to check their e-mail as millions of iPhone users clog their networks. Verizon’s network may be the biggest, but my guess is that users in major cities will find out that biggest and fastest are two completely different things.

You can send me my check now, AT&T.

Google saves the world five seconds at a time

Google just announced new technology that will cause search results to be predictively presented to the user as they type each letter of their search, saving the user from having to type their entire search, or hit enter after each search. According to Google’s Vice President, this will save the user an average of “two to five seconds…” from each search. This is what they have been working on as their next generation of search. It’s impressive, to be sure.

But does it strike anyone else as odd that we, as a society, are putting our technological efforts and resources into developing technologies that save us seconds each day when we can’t, as a society, get it together to develop infrastructure and transportation technology that will save us hours from commuting? It seems perverse that most Google engineers probably sit in California traffic for over an hour each day so that they can come into work and make sure nobody has to wait more than 15 seconds for a search result.

Moving…

In the coming months, I’m going to be leaving MIT. I’m not sure how long my access to my computer account will last, so I’m starting the process of moving everything over. For those of you that access this blog via an RSS feed, nothing will change. However, if you have this blog bookmarked, please change the bookmark to point to www.jonbirge.net. For now, that will still put you here. In the future, who knows?

Interesting article on high frequency trading patterns

The Atlantic recently carried a fascinating article on the odd patterns in stock market data caused by mysterious high frequency trading algorithms which post and remove bids and asks on stocks with no intention of ever buying the stock in question. My best personal theory is that these mysterious bids and asks are simply ways of “pinging” the market electronic infrastructure for information about the speed and timing of the hardware running it. High frequency trading has become so fast that firms do their best to but their computers as close as possible to the exchange’s servers.

My guess is these bid patterns are just acting as clocks. To quote one of the experts in the article, the speed of light actually starts to matter. Given that the timing of trades can mean millions in profits for some of these firms, it stands to reason they would try to keep close tabs on the speed of their connection to the exchanges. By sending in a sequence of false bids (bids so far away from the current price and retracted so quickly they will never result in a buy) they can safely figure out how long it takes an order to go from their computer to the exchange’s computer. They use a sequence (ramp) of increases prices simply to keep track of all the different orders and their timing. (If they used the same price repeatedly, they’d have no way of knowing which order was which.)

My next best theory is that they are probing the HFT algorithms of other firms, trying to either reverse engineer other trading programs (which would give them a decided advantage) or search for some way to induce other programs to act in a way that can be exploited, like a big robot battle played with real money. They are trying to see if they can predict what others’ programs will do in a way that doesn’t require them risking any money. Frankly, I don’t see this as too likely, given that it would be very poor programming for anybody to let their HFT algorithm respond to ludicrous bids.

A final, rather crazy possibility, is that these false bids are a way to communicate between market participants over public, non-traceable channels. If encrypted information were embedded in the timing or changes of the bids, it would be a good way for firms to collude in a way that would be nearly impossible to trace: everybody is allowed to post bids, and every is allowed to read them. Unless you knew how to decode the information encoded, you’d have no way of proving information was being exchanged between parties. And what better avenue of communication between financial firms than the stock market’s public quote system?

One thing that is certain is that this frenetic activity, with hundreds of quotes a second being generated and cancelled, is clearly meant for the “benefit” of other computers, either at the same firm or another. It’s a fascinating phenomenon. I’m not sure what it portends for the markets, however. As people leave the market in droves (as evidenced by 13 straight months of mutual fund outflows) I wonder what will happen when the only people left aren’t people.

First impressions of the iPhone 4

I had the honor of being the last person to get an iPhone 4 in Cambridge on launch day, wandering in to the Reserved line at 8:59, stupidly thinking the store closed at 9:30 instead of 9. Am I glad I braved the line and the self-loathing attendant with standing in line to buy a cell phone like some teenage girl camped out to buy Justin Bieber tickets? Yes. Yes I am. Below are my impressions on a few major elements of the phone.

Physical Build

This has the nicest feel to any consumer electronics device I’ve ever used, let alone a a phone. Using Corning Gorilla Glass for both the front and back was a beautiful touch. Not only does it look nice, but it resists fingerprints (due to oligophobic coatings which makes grease happier to stay on your finger than stick to the coating) and makes scratches nearly impossible. Early users who report scratches in the glass are likely seeing scratches in the coating, not the glass. While it’s a myth that only diamond can scratch glass, you’ll nonetheless have to try very hard to scratch Corning’s chemically hardened glass.

Another nice touch was going with a squared edge instead of the visually weak rounded edge of the prior generation. It’s like the difference between a ’91 Ford Taurus and ’65 Chevy ’67 Shelby GT. The phone has a solid heft, and the hard edges give it a more secure feel in the hand. (This will be especially important as you try to delicately hold the phone so as not to cause reception to drop, as I’ll describe below.) It has the feel of something substantial, that you might actually use for more than two years before throwing out.

Interface Feel

The A4 chip is impressive. The phone feel orders of magnitude faster than the 3G. No waiting for menus to come up. Everything is fluid and quick. They are going to save a bundle in post production when they do the iPhone 4 ads, as no special effects will be required to keep the ad under two minutes. (I’ve been tempted to make my own ad for the iPhone 3G where I do exactly what Apple does in their ads, except my ad will be shown in real time, with the announcer making awkward comments about the Met’s bullpen while he waits for the screens to come up and the keyboard to unfreeze.)

The iPhone 4 actually works as well in real life as the iPhone 3G does on TV.

Display

Shockingly vivid and sharp. I thought this was just going to be another phone, and I’d simply hoped it would (ironically) fix the reception problems I’ve had with my 3G. I had no idea how blown away I was going to be by the display. If you buy one of these phones, enjoy the feeling you get the first time you see it, because it’s going to be the last time you ever feel that way about a phone: despite the false controversy, these displays really are at the limit of what the human eye can see (unless you’re a 12 year old with 20-15 vision). There’s really no better they could do. Pictures look like slides placed on a light box. It’s quite an effect that you really have to see to appreciate.

Reception

Now, the ugly. The reception problems are true, and have already been acknowledged by Apple and Steve Jobs. Apple packed so much stuff into the inside of the phone that they had to put the antenna outside. The outer metal band of the phone is comprised of three distinct stainless steel segments that form the antenna used for (at least) cell and WiFi reception. (I’m not sure if the GPS antenna is internal, someone please let me know.) What this means is that when you hold the phone, you’re holding the antenna directly. Anybody whose ever touched a TV antenna knows that touching an antenna can affect the signal.

In this case, however, there is an even worse effect: if you hold the phone in your left hand (which you will if you’re right handed when using the touch screen with your right hand) your sweaty little palm will likely bridge two of the segments of the antenna. Since sweat conducts (due to dissolved salt) you will partially short the antenna. I can get the phone to drop from five bars to two just by holding it in a natural position. Even when not touching the phone, the reception is significantly worse than my wifes 3G phone, also on AT&T. I guess we’re in for another generation of iPhones that do everything well but make calls. Steve Job’s response to a user who e-mailed him about this was “A non issue. Just don’t hold it that way.” Yes, he actually said that. Even more amazingly, I’m guessing Apple will actually get away with this.

I can also tell you right now exactly what Apple is going to do about this: they will issue a software update in the near future that “fixes” the issue by artificially boosting the number of bars shown. This is what they did on the 3G, and people actually fell for it. Apparently having calls dropped while five bars are showing isn’t enough to raise any suspicion in the average Apple user. I have to sincerely admire a company that can achieve this kind of user loyalty. Steve Jobs is nearly ready to trade in the black mock turtleneck of a Sith apprentice for the hood of a master.

How did Walt Mossberg possibly miss the reception issues in his review? Robots don’t sweat.

Camera

The camera on the phone is so good that it’s probably going to replace my small Canon digital camera. Having a camera always with you that also geotags your photos is really nice, and something I never anticipated as a significant benefit of the iPhone. Nonetheless, I find myself taking a huge number of pictures with my 3G, because it’s just always there when my kid does something worth recording. So, at least for me, one significant justification for the upgrade is that Apple finally got the camera right on this model. While it’s not actually as good as a dedicated camera (even a small one) it’s close enough that it’s certainly worth not having to lug around a second device. Obviously, you’re not going to replace your DSLR with your iPhone, but when was the last time you, your SLR and something worth photographing all found themselves in the same place?

The iPhone 4 can also take 720p HD video. It’s not great quality, and is only at 23 frames per second. It has a slightly blurry quality to it, which I think is due to the detector being used at it’s full native resolution, rather than interpolated from a higher resolution. You can tell this is the case because the scene “zooms in” a bit when switching to video mode, which I believe is a technical limitation; there likely just isn’t enough camera transfer bandwidth available to allow for a full frame capture and then interpolation down to 720p, at least not in a way that wouldn’t require further reduction in the frame rate. I hoping there is a way to do lower resolution video at a higher frame rate, but I haven’t found it yet. HD video of any kind of pretty impressive for a cell phone, so it’s hard to complain about this at all.

Final Thoughts

Aside from the poor RF performance, it’s an amazing piece of engineering. You’ve got a high-end GPS chip, custom low-power processor, human resolution-limited display, broad spectrum LED backlight, a micro-machined gyroscope and accelerometer array, all clothed in chemically hardened glass package the size of a cigarette case. You basically have some of the most impressive modern optics, electronics, microtechnology and radio circuitry available, all in your hand. In fact, the RF problems stem from a design compromise they had to make in order to fit all of this in such a small package.

Having given them this much credit, however, one has to wonder about the wisdom of a design choice that puts size above reception on a cell phone. You know, a pebble is small, but it doesn’t get good reception, either. That’s why I don’t carry pebbles in my pocket even though they are incredibly portable. I’m willing to have a slightly larger phone if it actually works.

Will I keep the phone? I don’t know yet. I’ll have to see how well the phone works in problematic areas for reception, like MIT’s campus. I’ll also look at comparative download speeds between the 3G and the 4, to see how much the reception issue really affects things. However, for now I’m inclined to keep it and just tell myself that it’s really so much more than a phone, how can I expect it to make calls? Steve Jobs really knows what he’s doing…

Update: I recently did a download speed test. Holding the phone normally: 0.2 MB/s. Holding it with the tips of three fingers: over 2 MB/s. This is repeatable.

iOS 4 significantly slows down the iPhone 3G

Just a quick post to warm people that the iOS 4.0 upgrade will significantly slow down an iPhone 3G. I don’t know if it’s the slower processor or the lack of RAM, but I very much regret making the upgrade. The only useful features of iOS 4 that are enabled on the 3G are folders and the new version of Mail. While those are nice, they don’t begin to make up for the incredibly slowness of the update for certain tasks. Typing in addresses in Mail, for example, often hangs the phone for several seconds. It now takes a few seconds for the settings menu to first come up, as well.

Does the world really need MBAs?

A few weeks ago I was sitting in a coffee shop writing my thesis. Next to me were two students from Harvard’s Business School, that esteemed institution responsible for many of the managers who have been doing such a bang up job of running our nation’s financial system. They were going over a case for one of their classes, and from listening to them struggle with simple math, it was apparent that those two, who are at the best business school in the country, probably wouldn’t have lasted a minute in any graduate program in the hard sciences.

To begin with, I am skeptical of the very idea of having a management class that swoops in from their MBA school, sans real world experience, to manage companies. It seems to me it would make more sense to pick management from people inside a business who have demonstrated understanding and ability of the unique aspects of that industry.

From my admittedly distant vantage point, business school seems to train people not to truly lead, but to be the quickest in following the herd. Instead of giving us managers who learn a business and find ways to improve its product, it gives us a bunch of jar scrapers who do little of sustance but simply find increasingly clever ways to fool people into paying more for less using the latest management fad. You leverage your customer base into a value-added service relationship, or you outsource non-core competencies, or synergize across divisions, etc. Anything to goose the next quarter earnings growth. You put Snickers bars near the check out at Kinkos, but god forbid do you actually innovate and create something of unique, sustainable value.

Of course, you can’t expect them to do that, because true leadership isn’t something you pick up in a seminar, and acquiring the skills to really innovate in a meaningful way requires a tough slog through an actual technical education. Don’t get me wrong, I have tremendous respect for good managers, having worked under a few. But it can’t be taught in a class. Worse, the concept of business school means that our leadership class is an entirely self-selected group. Leadership should be a position that is earned, not self-anointed by one’s choice of graduate school.

In fact, give some thought to what kind of person even thinks it possible to become a valuable leader with two years of night school, and you begin to understand why corporate America is the fix it’s in. It’s no surprise so many of them cut corners ethically and focus on short term results at the expense of true sustainable value. When I was applying to grad school, I looked at the statistics of the GRE scores by major. Business majors scored lower than everybody; lower on Math than English majors, and lower on Verbal than Engineers. It seems the only qualification one has to have for leading people in corporate America is an inability to do much else.

This kind of pseudo leadership is epitomized by the well-documented decline of Hewlett-Packard under the erstwhile tenure of Carly Fiorina. This once great innovator that gave us the first handheld calculator was morphed into a company that sells cheap plastic printers at cost so that they can gouge consumers with ink at 1000% profit margins. Short term, that works as a way to make money. Long term, HP is never going to invent anything again, because after all the short-sighted cost cutting, their research labs are essentially defunct. Of course, a marketing consulting firm was probably paid hundreds of thousands of dollars to come up with their “invent” slogan. The management class has no sense of their own self-created irony.

Like HP, many once great American businesses are essentially management consulting companies that happen to have inherited some intellectual property. Far out product development funding is decimated, and what remains is for management to scrape the bottom of the jar with marketing tactics, outsourcing, and whatever trivial refinements are allowed by their skeleton R&D departments. Looking naively at corporate profit growth in the US, the MBAs seem to be vindicated, but this growth has been illusory. They call it transforming America to a knowledge-based economy. I call it burning the furniture to heat the house. How much longer can we grow by shrinking?

Given the ubiquity of MBAs, it’s easy to forget that the very concept of business school didn’t come about until the early 20th century, and it didn’t take off until after the war. We managed, as a country, to produce some of the world’s greatest industrial achievements without the aid of MBAs. We built a transcontinental railroad, gave the world aviation, invented the automobile industry and modern assembly line manufacturing, all without a single business school graduate around to synergize or value-add anything.

I believe history will show that the concept of management school, and the notion of a management class that is self-selected by career choice and not demonstrated ability in a field, is a major failure. Maybe it’s time to rethink our pipeline for corporate management.

Obviously, any time one is talking about an entire group consisting of hundreds of thousands of people, you’re talking in approximations and on the average. Some of the great leaders I’ve met that I alluded to above actually had MBAs. My point is that they are great leaders not because they have MBAs, but because of their experiences and inclinations. In fact, the people I know who I respect the most with MBAs have very little respect themselves for the degree, which is where much of my skepticism about the degree originates.