Via a registered investment advisor, who explains; You will want to read this, which explains exactly [the flash crash] a year ago May 6th. … If the exchanges allowed this argument to enter the mainstream press, the US stock market would be closed within a week. They are that dependent upon ‘flow’ to survive.”
Some algorithms, for example, can detect the electronic signature of a big VWAP, a process called ‘algo-sniffing’. This can earn its owner substantial sums: if the VWAP is programmed to buy a particular corporation’s shares, the algo-sniffing program will buy those shares faster than the VWAP, then sell them to it at a profit. Algo-sniffing often makes users of VWAPs and other execution algorithms furious: they condemn it as unfair, and there is a growing business in adding ‘anti-gaming’ features to execution algorithms to make it harder to detect and exploit them …
Whatever view one takes on its ethics, algo-sniffing is indisputably legal. More dubious in that respect is a set of strategies that seek deliberately to fool other algorithms. An example is ‘layering’ or ‘spoofing’. A spoofer might, for instance, buy a block of shares and then issue a large number of buy orders for the same shares at prices just fractions below the current market price. Other algorithms and human traders would then see far more orders to buy the shares in question than orders to sell them, and be likely to conclude that their price was going to rise. They might then buy the shares themselves, causing the price to rise. When it did so, the spoofer would cancel its buy orders and sell the shares it held at a profit. It’s very hard to determine just how much of this kind of thing goes on, but it certainly happens. In October 2008, for example, the London Stock Exchange imposed a £35,000 penalty on a firm (its name has not been disclosed) for spoofing.
Still, it’s probably nothing.
Update: From the comments…
I’m in software.
Years ago, I worked for a company that provided control software to the US Air Force. There were lots of papers to sign acknowledging the importance of “secrets” and we all had to sign waivers saying that we understood the penalty for divulging any material infomormation could include imprisonment in any NATO country…etc.
About two years ago, I interviewed for a job at a company that does nothing but high-speed trading algorithms.
The trading company was far more strict (and menacing) when it came to secrecy. It actually felt to me like they were some sort of cult.
I went through two interviews and, even at that point, they wouldn’t discuss the type of work they did. Instead, they’d just hint that it had something to do with the stock market. I knew who they were because a friend of mine had been through their grinder. But it was surreal…they wouldn’t even acknowledge that certain people had previously worked there.
Heck – at the end of the second interview, they weren’t even willing to name the job positions for which they were hiring.
Even the US Air Force wasn’t that super-secret.
I understand that these companies typically pay tens of millions just to buy old, plain looking brick buildings at “strategic” locations in New York, New Jersey, and Chicago – just to be physically close to the backbone of the systems on which the trades are conducted … thereby trying to gain microseconds of transaction speed over the competition.
Whatever their real impact on the markets, I am quite sure that these companies certainly see themselves as some sort of secret society.
I have no trouble believing that they have the ability to cause a flash crash. Honestly, I’m surprised we haven’t seen worse – and more often.
Finally, something that explains the completely insane action on the markets. Cheating! Makes perfect sense.
Also, it makes sense, -finally-, of why Big Wall Street supports the DemocRats against their own best interests. The DemocRats are the party of liars and cheaters. QED.
How about we just take people completely out of the equation and let algorithms decide our futures?
I’m in software.
Years ago, I worked for a company that provided control software to the US Air Force. There were lots of papers to sign acknowledging the importance of “secrets” and we all had to sign waivers saying that we understood the penalty for divulging any material infomormation could include imprisonment in any NATO country…etc.
About two years ago, I interviewed for a job at a company that does nothing but high-speed trading algorithms.
The trading company was far more strict (and menacing) when it came to secrecy. It actually felt to me like they were some sort of cult.
I went through two interviews and, even at that point, they wouldn’t discuss the type of work they did. Instead, they’d just hint that it had something to do with the stock market. I knew who they were because a friend of mine had been through their grinder. But it was surreal…they wouldn’t even acknowledge that certain people had previously worked there.
Heck – at the end of the second interview, they weren’t even willing to name the job positions for which they were hiring.
Even the US Air Force wasn’t that super-secret.
I understand that these companies typically pay tens of millions just to buy old, plain looking brick buildings at “strategic” locations in New York, New Jersey, and Chicago – just to be physically close to the backbone of the systems on which the trades are conducted…thereby trying to gain microseconds of transaction speed over the competition.
Whatever their real impact on the markets, I am quite sure that these companies certainly see themselves as some sort of secret society.
I have no trouble believing that they have the ability to cause a flash crash. Honestly, I’m surprised we haven’t seen worse – and more often.
Let a flash crash happen, have a few money mangers and traders jump from the buildings and sanity will reassert itself.
I don’t think that computer generated algorithmic information processing is ‘evil aliens controlling us’.
The fact is, that information processing by humans, by any and all organisms, even the microscopic, is a basic component of life. And it’s done the same way: algorithmically, which is to say, calculating a function or interaction and coming to a decision based on received information.
The concern is a supposition that IF it is ‘automatic’ THEN it is destructive. This is a leftover from simplistic darwinism where information processing is random and heavily destructive…and only a few gasping heroic survive.
The real world – human and otherwise- isn’t that random. The premise I would use is:
Is the information gathering by these algorithmic methods within an economic system geared to destroy that economic system? My premise is that: it isn’t self-destructive.
But the process is larger than individuals – individual traders – and if individuals interfere or ‘cheat’ at various points, this throws the process off to a deviant path.
So, the next concern is a supposition that this process can be taken over by unethical agendas. That’s possible anytime and everywhere; it’s how humans operate. Unethically.
One security tactic would be to introduce a method that outlines interference on the normative path. That includes ‘spoofing’ which is a serious interference. I’d say a fine ought to be much more than the meagre amount imposed by the London Stock Exchange.
For the last several years I’ve been following a webs site that focuses exclusively on “naked” short selling and the chaos it causes for civilian investors. The current story at deepcapture.com reads like a cross betweeen John Le Carre and Mario Puzo. A confluence of organized crime, jihadis and nation-state intelligence services playing the markets like a mandolin in an organized attack to bring the USA to its knees.
The guy who funds the site literally grew up on Warren Buffet’s knee and is no “outsider” to high finance or trading…he runs an internet based “Amazon” type firm called Overstock.com. If a tenth of what is posted is accurate it calls for a serious criminal investigation.
This also explains the cheating on climate modelling – they’re using Al Gore-ithms
The payments the exchanges receive for allowing the servers to be next to the exchange’s own servers are enormous. Imagine placing your own, faster, server next to the exchange’s and connecting them with a three inch diameter, foot long fiber optic cable.
The practice, however, is known as co-location.
ET-
That answer proves you didn’t bother to follow the link.
It looks like an Al Gore Rhythm.
melinda – actually, I did follow the link and I don’t agree with the supposition that algorithms in one system, without outside human interference are fighting with each other.
That’s what I meant by suggesting that such a system, without the interference, would not self-destruct. It’s like ‘genetically modified plants’..that, if changed without thought for their interactions with the envt, harm both the plant and the ecological domain.
Ah, but the sub-cycles of sub-penny quote stuffing is how the different algos go after one another, and then front run an actual trade. They stuff each other.
If they really don’t want to participate, the algos are written to present “stub quotes” where the prices displayed are so far away from the “real” market so they don’t participate in the trade.
Go here to see another example of what stub quotes can do to a market: http://www.zerohedge.com/article/presenting-natgas-fractal-algo
And I apologize for accusing you of not reading that.
HFT traders pay the NYSE to locate their servers on-site, thereby getting all data coming and going before anyone else. They don’t need nearby buildings … they are located right in the exchange.
melinda- I think we are talking past each other.
I am talking about algorithmic processes that are NOT interfered with. You are talking about those that ARE interfered with…such as quote stuffing.
And I suggest that a code could be added to the algorithmic process that ‘sniffs out’ these bogus trades. That is, the computer, the machine, is not going to take over the world.
Buy real gold and silver. Seriously. It is all coming down real soon.
My thoughts are that the Stock Exchange regulators are far too slow in coming to the party. If this is true, they should be on top of things before any private snoopers are. They certainly burn up enough resources for that kind of performance.
What I think we are seeing here is the all to human phobia of ROBOTS that Issac Isamov wrote so much about….it is a very real and very primal fear.
We have had to more or less trust robots for some time.
Naval carrier aircraft have been landing this way for some time…..capable of putting a multi-ton aircraft onto the 3rd wire in the dark, fog on a pitching deck….reliably….not that the pilots really enjoy it…..workers in factories with robots are careful to stay beyond the yellow line.
This robotic trading is distressing because we can’t see the yellow line or be sure there is a yellow line.
why all the whining? this is capitalism at its defining moment.
the capitalism ALL Conservatives hold in highest regard; you followers of Milton Freidman.
no time for 2nd thoughts or 2nd guessing.
Ping: Clearly, you have no idea what Freidman taught?
Cheating, even if you pay to do so, was never in his conceptualizations.
To him, Capitalism was like a sport, which needed rules that were fair and “equally” applied. To “progressives”, Capitalism is a means to power, whereby rules are to be manipulated so that the deck can be stacked in favor of chosen ones. (note, that the GOP and Canadian CPC are loaded with “progressives” … not just leftist orgs)
A free-for-all, without law and order and rules, is not capitalism … the “free-for-all” is an invention of leftists who want to misrepresent what capitalists believe. As you have done in such a ham-fisted way.
It reminds me of the movie “Traffic” where Michael Douglas (I think) was the US official tasked with fighting the ‘War on Drugs’. MD was stunned, as were the viewers of the movie, to find-out that the drug cartels had 10x the resources the US government had to conduct the WoD.
This is the same situation, where the “crooks” or in this case trading companies will ALWAYS be miles ahead of the laws, penalties, and security.
Yeah, sure Pingo and Liberalism is that “caring sharing of capitalism’s cash”, after they crook it from “us”. You’re somewhat a better choice, eh?
The only difference from Conservatives, as you portray them is that Liberals do the same things, but do it to benefit themselves and their “friends”. “We” still get crooked, just by a more “caring sharing” bunch of crooks, who decide for “us” who benefits and then charge “us” your 17.5% “commission” and stick it in your pocket.
Why don’t you just FO!
ping; Capitalism = money and a reason for hope. Socialism = despare and misery.There is a book pulished extolling the advances for humanity that socialism has brought to society.It consists of 600 plus blank pages.There are only 500 blank pages but they represent 600 plus pages as the author went green.
What if these ROBOTS were programmed with a purpose other than for making money? What if they were programmed to crash the financial system in a coordinated attack? What REALLY happened in the crash of 2008? One theory in a US DoD report is that 2008 was the first part of Financial Terrorism attack. It would appear that these ROBOTS would be a perfect weapon for these attacks.
Check it out:
Financial terrorism suspected in 2008 economic crash
http://www.washingtontimes.com/news/2011/feb/28/financial-terrorism-suspected-in-08-economic-crash/?page=1
ET-
No, these are not interfered with in the least, build in a sinewave function, with step gains on the amplitude, and you can sniff out anything, automatically, if another bot is running in that security.
Pretty simple stuff, mathematically.
Another example: http://www.nanex.net/StrangeDays/06082011.html
Short trading should be illegal, those who profit from short trading are the ones who do the most damage to our markets-my opinion only.
I think the lesson to take home here is that these traders are building a little ecosystem of competing algorithms, they have NO idea at all what the long term outcome will be, and the regulators are ten steps behind them with even less clues than the traders.
Plus, the regulators have their own agenda, which is not necessarily the continued functionality of the market.
Ultimately the more complex a system gets the less it takes to break it. What continues to amaze me is that this thing keeps working.
Rose, “shorting” is just selling a call or buying a put, these are offers to buy or sell a certain stock at a certain price higher or lower than the trading price of the moment. They are not evil in and of themselves, just confusing. If you think a stock is going to go down, you can make money by defraying some of the risk from people trying to buy or sell it.
Its a service.
Naked shorting is fine, as long as you have the dinero to back up your contracts, just like anything else. If you don’t, then you go to jail for fraud.
I understand that these companies typically pay tens of millions just to buy old, plain looking brick buildings at “strategic” locations in New York, New Jersey, and Chicago – just to be physically close to the backbone of the systems on which the trades are conducted … thereby trying to gain microseconds of transaction speed over the competition.
I’ve heard of that, too, and demands that their hosted stuff be on the shortest cables possible (speed of light!), and definitely no longer than the competition.
The hilarious part, however, is that despite their earnest belief that those picoseconds and microseconds matter, I can’t find any reason to believe they actually do.
Much more like a cult than like a science.
(I’m much with Phantom Idiot – and I am likewise somewhat amazed that these houses of cards based on ridiculous fetishization of “instant trades” don’t just collapse on their own.
Seems like they’re really missing the point of how arbitrage is supposed to work.)
Rose: I can see an argument for naked shorts being more regulated (though frankly I also don’t think they cause any real damage).
But shorting something you can actually cover is a price-correcting mechanism, just the same as going long is.
If you ban the signals of the market as to predicted future value, you don’t save it – you destroy it.
melinda – a sinewave is a regular, normative pattern with an expected variational range. Quote stuffing is an external non-normative interference in a normative process.
It’s like a tsunami-earthquake – which is a non-normative process caused by external force. So, the ocean reacts…
In the markets, if someone deliberately interferes with the normative range (in both time and space) of buying and selling…the algorithm isn’t picking up that this is an external agent deliberately interfering with the market and outside of its normal range of variation.
That’s my point. I don’t see an algorithm taking on an agential role to move into a non-normative mode of behaviour; that’s due to an external agential force. But once an external agent moves in to create a Non-Normative pattern then, uncertainty both via external agents and within the system can take over and crash the whole system.
That’s what happens, for example, when an external agent, a disease, moves into a physical body.
After all, the fact that ‘spoofing’ is caught means that there IS a normative pattern.
ping- this has absolutely nothing to do with capitalism. Or socialism. It has to do with ‘being a crook’…and if you think that there are no crooks in a socialist economy, well..
cjunk has outlined it very well.
And it’s Friedman not Freidman. And he refers, often, to laws and regulations.
I’d say sigivald and phantom are either utterly ignorant of the staggering damage naked shorts inflict and the tremendous opportunity it gives crooks, or they are part of the “sock puppet” army who spend untold hours pumping out lies and disinformation. Naked shorting is Counterfeiting pure and simple. It is market distorting con game…and is completely against the law 72 hours after a trade fails to deliver. End of discussion. Shorting is a valid market exercise, naked shorting is a crime.
Bit of a side note: I’ve worked in the software industry for nearly twenty years, and I am routinely amazed that any software company stays in business more than six months. The rampant incompetence among both programmers and management astounds me daily – and this in an industry where it’s actually possible to mathematically prove whether you’re doing the job properly.
So none of this “shorter wire” nonsense surprises me. I’ve seen tech CEOs spend just as much money on completely ludicrous things. The company I work for just spent a couple million to develop, internally, a clone of Facebook for employees to use. As far as anyone can tell, it has absolutely no work-related function and no one uses it.
As for sniffing out spoofing algorithms, that’s about as hard as writing a 100% effective email spam filter.
For those of us who follow both SDA and zerohedge, this is old news.
Bryceman, I hear you. A year or so ago, some Russian programmer was arrested on charges related to divulging algo secrets; when real money is involved, such as this, the big guns come out.
ET:
I respect your normally intelligent and lucid comments, but you’ve missed the boat here, by many lengths. For over 100 years, the expectation in financial markets is that bids and offers are 1) real, 2) unmasked, and 3) representative of the market. Algo trades are none of these. As ZH has documented over and over, these “offers” are only valid for milliseconds, and if any one tries to actually make a trade, the offer is immediately voided. In “open outcry” markets (I commend to you the climactic scene in Trading Places, where Dan Akroyd and Eddie Murphy use comic hyperbole to great affect), this is not possible. Open outcry makes for transparent markets, where the capitalistic ideal of all information available to all participants (as far as bids/offers/interest goes) is almost achieved.
I have ceased all trading on the NYSE/Nasdaq since the advent of algo trading. If it comes to Toronto in a big way, I won’t trade there either. Either you have a marketplace that’s fair for everyone, or you have a rigged casino. That’s my assessment of Wall St. today, and anyone who trades there is a fool, IMHO.
kevinb – I accept your criticism; it’s valid and the field of trading is not my area. My interest is in the ‘informative networking of algorithmic interaction’, understanding an algorithm as a function.
I’m suggesting that this process occurs naturally in chemical and biological processes. That is, the interaction is a normative algorithm interacting with the envt that allows for certain ranges of adaptive robustness.
Now, looking at societies – which are grounded in economic transactions – what goes on when economic transactions are moved from direct individual actions (macroactions) and instead, moved to ‘normative patterns of a network’ of actions (microactions). Via a computer algorithm.
Can one anticipate the patterns within such micro interactions? And what about ‘rogue events’?
I know of several research projects focused in this area – i.e., in the ‘interface’ zone where networked electronic information is processed and an output is generated. In economics, in marine and ship operations, in markets. These are research projects not solutions.
My curiosity is on the scope as well as the limits of such a process; will it go rogue, are there constraints within the whole process of information processing…and..are these ‘natural’ or must they be built in by an external regulator. Importantly – Can such a system learn, on its own, and develop new interactions and solutions?
I don’t think that robotic actions will take over the world – or economics. I think that these methods and in particular, the interface where information is exchanged and yes, transformed – are fascinating.
ET, I think the danger in a situation like this is that the activity by algorithms will skew the market. The churn and thrash of these things competing with each other on a millisecond basis can mask the “real” price signals of supply, demand, greed and fear that usually govern stock prices.
It isn’t going to evolve into Skynet though. That’s a little too far to stretch.
The economy as a whole would be far better off if such ‘speculative’ trading were banned. The idea that without improvement or effort on my part I can sell to you tomorrow what I bought from you yesterday and make money doing so does not benefit the economy. The fact that people have developed algorithms to do such faster and more easily is even less helpful in making the economy tick.
OK, ET, I get your drift.
Here’s the limiter that I should have been more explicit in outlining.
Normative markets for large cap stocks are two pennies wide, say, 39.01 bid, offered at 39.02. Those are the makets presented to the public and are the only prices where average citizens are allowed to participate. “Liquidity Providers”, aka HFT systems, may trade sub-penny. They show bids and offers at 4 and 5 decimal points, depending on the stock. Where one algo game kicks in is as follows, and it IS known as quote stuffing, from 39.0101 to 39.0199 a program is written to show a bid for 10,000 shares every 1/60,000 of a second, upticking the bid from 39.0101 to 39.0199 and then restarting, barring an actual order. This, obviously, would show up on an oscilloscope as a right triangle signal, and any legitimate order in that cycle would show up as a perturbation of the triangle. Skew right, it’s a sell order, skew left, it’s a buy order, and it would take no real effort to pre-program the system to step in front of the actual order.
These algos are creating the normative, as you describe it, not acting within it. A big difference. This is not an algo written to learn and win, it’s written to exploit the system’s weaknesses.
The rogueness is the flash crash, when the defense parameters kick in and the stub quotes actually get traded on.
Much like court evidence will only accept polaroid pictures, I think computer trading should be eliminated and the only acceptable method should be ticker tape. Kind of like the Senate, useless but a lot of time to think about the subject matter. Think of the benefits. No more wild swings. A few weeks to see if your stock moves up or down is far less pressure on traders. Short traders would have to wait long enough to gain a conscience. Speculators could have something to hold when they jump off a building and many many more advantages to make the stock market fair and transparent. Shit would still happen but at a slow and enjoyable rate.
Yup…computers are the problem and we could show we are serious by stringing up Bill Gates. If he had stuck to windows 3.1 we would’nt be in this mess and talking babies would not have computers in their cribs.
Kate: please forgive this marginally on-topic comment.
Trolls, please notice that in this thread, people like ET, Melinda, and myself are able to disagree, and discuss an issue without resorting to insults and baseless accusations. We are actually engaged in discussing something in the hopes of learning and increasing understanding, as opposed to drive-by calumny.
Stock markets have always been places where masters of manipulation sought to outfox each other. We’ve had Enron, Madoff, and Subprime mortages recently, and you can find examples all the way back to the South Sea Bubble (1719) and the Dutch Tulip Mania (1630s).
What’s different here is the scale, and the rate of change. Things change so fast that people can’t react to them in time. Should someone build a better algorithm, they might be able to take out several other banks — take all their money, that is — before the other banks figured out what happened to them. That’s sort of what happened in the Subprime mortgage scandal.
If this technological arms race is able to keep a rough balance between major competitors, on the other hand, it might end up making things more stable than it has been recently. But the system is still quite fragile.
The best description of the problem I’ve seen was on the old Outer Limits show, and it goes to the heart of the unprecedented times we find ourselves in.
“If knowledge is power, and power corrupts, how will humanity ever survive?”
New algorithm is OLD INSIDER KNOWLEDGE WRIT LARGE.
The real problem here is that everything happens in a non-human time frame … much as CAGW on the other end.
Anyone that thinks an algo war means anything but continuing and escalating problems has never programmed anything to act, or counteract, a changing something.
Thanks Melinda … learned a lot from you. But, I would change “it’s written to exploit the system’s weaknesses” to ” it’s written to exploit any system’s weaknesses”.
Melinda, at what point does the action of these things like quote stuffing drown out the real world? Or can that even happen?
Wraith(Phantom)-
With the NYSE volume averaging just over 3 billion shares a day, I’ve been hearing estimates between 70 to 85% of the trades are HFT driven. I wouldn’t be shocked if that’s true.
So that would be the real world, for us.
Don’t get me started on “dark pools”.
I thought “algorithm” was a birth control technique for global warmists?