A bank client found itself in a world of hurt after a lending officer detected and exploited a defect in the bank’s loan accounting software. The lending officer “lent” money to his accomplices far in excess of the bank’s dollar limit for individual loans.

When the scheme collapsed, the bank sued the banker and his accomplices. The bank should have sued the software vendor. The software was written to enable lending. It should also have been designed to flag the obvious risk of over-lending to individual borrowers. This defect was a product failure, no less than the software failures that crashed Boeing 737 Max aircraft recently.

For both the bank and Boeing, the process that went haywire was standard operating procedure: manufacturing loans at the bank or flying commercial aircraft at Boeing. Along the way, people got sloppy. They met sales goals and delivery date targets at the expense of serving mission critical objectives. The difference is we do not think of bank software-induced crises as being products liability cases in the way we do Boeing’s failure. We should.

Given the regulatory regimes that apply to banks and commercial aircraft, it is tempting to look there for a solution—to regulate bringing financial software to market the way the Federal Aviation Administration does commercial aircraft. Public pressure for that result is growing, framed mostly in terms of protecting consumer privacy.[1] The Boeing story though is a reminder that regulators can be fooled no less than end users. And the last century’s experience of regulatory capture by the largest companies in regulated industries reminds us regulation can stifle innovation and useful competition.

Our view is that financial companies can and should face the problem head on. Here are a few measures that are available within the resource set most financial companies have at hand.

1. “Think outside the box” is a cliché. It is also a useful discipline.

Just published, A Game of Birds and Wolves[2] recounts how a cadre of college age women in WW II England helped keep the nation’s citizens from starving in the face of German U-boat attacks on supply ship convoys crossing the North Atlantic. As tonnage losses mounted and economic collapse loomed, the Admiralty tasked a disabled officer and dozens of women serving in the naval auxiliary to study the problem. The Wrens (as the women called their group) created a theatre-sized board game that allowed them to replay sea battles based on after action reports. To represent the North Atlantic, they painted a grid on the floor of the large room they used for their work. They marked individual ship movements with chalk. They made game players—admirals, merchant marine captains and others—stand behind screens with slits cut in them, replicating the view from the bridge of a convoy ship. The players then called out moves of their ships as the battle evolved on the floor in front of them.

The Wrens discovered the Allied command misunderstood German tactics. Navy men believed U-boats were attacking from outside a convoy’s perimeter. Positioned along the perimeter, each Navy destroyer ship would chase any U-boat it spotted and fire depth charges in an effort to sink it.

In actual practice, German U-boats attacked en masse from the rear of a convoy, surfaced within it, fired torpedoes at point blank range, then dove to allow the convoy to pass over the U-boats’ positions. Upon making this discovery, the Wrens redesigned the Navy’s tactical response. Destroyers were instructed to break from the convoy in a group and mount a coordinated counterattack. The destroyers’ routes were plotted and tracked on a grid to cover the entire area where the U-boats likely were hiding. When the tactic worked and cargo tonnage losses declined, skeptical Navy men became believers and the U-boat threat receded.

In business, taking that sort of “let’s step back and think about it” approach is not an accounting exercise. Or a legal one. It’s a matter of imagining alternative, possible events rather than considering only those judged to be actual or likely. In the realm of software, it requires not only being excited by what newly purchased software can do, but asking what needs to be done that the software does not do, or does not do as well as we need it to be done?

2. “Just because the software says it, does not make it so.” Beware false positives, and identify where, how and why they arise. Published last week on the Internet was a video of a man pulling a child’s red wagon loaded with 99 smart phones down a three lane street in Berlin, Germany. The video showed almost no vehicular traffic. Yet Google Maps showed extreme traffic congestion. We all take software-generated output as gospel. We need to be appropriately skeptical.

3. Proper risk management requires rethinking a company’s approach to negotiating software licenses. Map and view the topology of all company software. Create a licensing template of acceptable business and legal terms, and follow it. Escrows and reserves for negative IT surprises should be the norm, no less than for credit losses. Do not blithely cave in to warranty and damages limitations in software contracts. Financial companies have standards and procedures for everything from colors used in advertising to financial reporting. Why is software any different?

4. Think expansively about relationships you can use for help when you encounter a problem. Verizon told a bank client it would be “a couple days” before Verizon could address a software problem that threatened to shut down the client’s branch banking business. I telephoned a former partner who had also been a state public utility commissioner. Within three hours the problem was solved. The client did not know who I knew when it called me; but the client though enough to ask what I might be able to do to assist.

5. Do think about products liability law to redress software failures.[3] To have a viable case, one must have negotiated rights and remedies defensively, as suggested in paragraph 2 above. But that is not a bridge too far by any means. The 20th Century saw American law move from permitting financial recoveries only when one could prove negligence, to a rule of strict liability in tort in some cases. The shift came because industrial products and the stream of commerce became so complex that it was no longer possible to prove negligence.[4] Software will increasingly be subjected to the same legal standards that apply in the realm of complex manufactured goods like automobiles.

As software continues to become more mission-critical in financial companies, failures will be more complex and damage more widespread. In early January, multiple large U.K. and European banks curtailed foreign exchange operations after money transmitter Travelex was crippled by a ransomware attack. One expert observed, “If there was ever any doubt that a cyber attack could have a significant effect on financial markets, this proves otherwise.”[5]

Up next: What do new developments in electronic financial services mean for the Board of Directors’ evolving “duty of oversight”?

[1] “The Government Uses ‘Near Perfect Surveillance’ Data on Americans--Congressional hearings are urgently needed to address location tracking,” https://www.nytimes.com/2020/02/07/opinion/dhs-cell-phone-tracking.html.

[2] S. Parkin, A Game of Birds and Wolves: The Ingenious Women Whose Secret Board Game Helped Win World War II (2020).

[3] For an early argument that software should be treated the same as industrial products for legal liability purposes, see https://resources.sei.cmu.edu/asset_files/TechnicalReport/1993_005_001_16187.pdf.

[4] When Gladys Escola stocked bottles of Coke in the refrigerator at the restaurant where she worked as a waitress, one bottle exploded in her hand, injuring her severely. The California Supreme Court said she deserved to be compensated for her loss even though she could not prove the Coca Cola bottler was negligent when it packaged the soda in the bottle. Escola v. Coca Cola Bottling Co., 24 Cal. 2d 453 (1944).

[5] https://www.insurancejournal.com/news/international/2020/01/08/553885.htm