Without question, the rise of social networks has been the dominant theme in Silicon Valley over the past few years. Platforms like Facebook and Twitter have inspired countless startups looking to latch on to networks to deliver new applications and services for consumers. In many ways, the glue that binds these enterprises is an advanced ability to organize and analyze the reams of user data generated by these networks or systems. Entirely new business models have emerged to try and capitalize on this improved understanding of consumer preferences and behavior.

Over the last couple of years, the analytics experts in Silicon Valley have started to turn their attention to other big data problems. A question that is increasingly attracting their attention is: How can the fallout from the subprime mortgage crisis be better managed for all the players involved, including at-risk homeowners, lenders, mortgage servicers and investors?

We’ve heard a lot about the near-universal frustration that at-risk borrowers have had with their mortgage servicers. The common refrain is that if mortgage servicers could only make smarter and quicker decisions on how to modify the terms of individual mortgages, then there would be fewer foreclosures on the margin and lenders or mortgage investors would actually lose less money in aggregate, since the foreclosure process itself is costly.

For many, the challenges in this area are about asymmetries of information or structural market frictions, since win-win outcomes aren’t realized as often as they should be in an otherwise efficient marketplace. Glenn Hubbard and Chris Mayer at Columbia University have developed plans for addressing some of the frictions that have blocked borrowers from taking advantage of today’s low interest rates by refinancing their mortgages. But now new companies, some with their roots firmly established in Silicon Valley, are eyeing the mortgage servicing market as fertile ground for deploying their creative and analytical firepower.

A prime example is Palantir Technologies (pronounced Pal-an-TEER). At its core, Palantir develops platforms that help other companies integrate and analyze their data. Initially Palantir’s focus was on the intelligence and defense community, helping organizations like the CIA and FBI ferret out terrorist activities. Analogous platforms have since been developed to help financial institutions comb through their networks to identify suspicious or fraudulent transactions. Hedge funds, including one of the world’s largest – Bridgewater Associates LP – have also knocked on Palantir’s doors looking for ways to leverage their open-ended or extendable platform as a way to better process or integrate their investment-related data and research, which often comes from multiple sources.

Joe Lonsdale, co-founder of Palantir, and Rosco Hill, one of his colleagues, recently gave a TEDx New Wall Street presentation about how this Silicon Valley-to-Wall Street workstream is evolving and has the potential to improve mortgage servicing. One of the underappreciated problems in the mortgage market today is that most servicers are still playing catch-up from when the housing bubble burst and their systems started to get overloaded with processing non-performing mortgages. A top U.S. Government regulator on housing – the Federal Housing Finance Agency (FHFA) – has even launched an initiative to restructure the way that mortgage servicers are compensated to try to establish a more durable servicing industry that is better prepared for boom-and-bust cycles.

The activities and costs associated with servicing a performing versus a non-performing mortgage are fairly dramatic. Before the housing crisis, when home prices were rising and foreclosure levels were down, the servicing industry was primarily thought of as a payments processing business (i.e., sending out forms to borrowers, collecting payments and passing cash flows on to lenders or investors). The best servicers were the most efficient processors, looking at each turn for new ways to streamline their systems to reduce costs and achieve economies of scale as a way to maximize returns.

Servicing non-performing mortgages, however, is a labor-intensive business. It can be difficult if not impossible to achieve economies of scale, since many mortgage workouts or modification strategies involve direct or personal interactions with borrowers. The underinvestment in servicing technology heading into the housing crisis was perhaps best summarized by FHFA:

Prior to 2007, servicers were mainly focused on building efficiencies in the servicing of performing loans in order to reduce costs and optimize financial returns. Relatively few chose to invest in the technology, systems, infrastructure and staff needed to service large or rapidly growing volumes of non-performing loans. Consequently, many servicers were ill-prepared to efficiently process the high numbers of delinquencies that occurred after the housing market collapsed. Since then, the servicing industry has increased its investment in the processes and technologies needed to meet the challenge of servicing non-performing loans in today’s environment.

While the five big banks that dominate the servicing industry (Wells Fargo, Bank of America, Citigroup, JPMorgan Chase and Ally Financial) have increased their investments in servicing-related technologies and infrastructure over the past few years, smaller servicers are also now looking to gain market share. Part of this emerging story is about the new data platforms that special servicers are utilizing to distinguish themselves from some of their competitors.

One area where new technologies are starting to make a difference is in helping servicers approve short- sale transactions as an alternative to foreclosure. (A short sale is when the lender accepts less than the full amount owed on the debt in a sales transaction, but still releases its claim on the underlying real estate.) The promise of new technology platforms on this front is that they can connect different data sets on home prices and other variables, whereas many big bank servicing platforms still rely on closed systems that don’t easily integrate all the public and proprietary data sources that are available. Better data integration allows for more comprehensive search and discovery processes, which have the potential to help servicers confirm what exactly is a fair value price for a home in a declining market.

The ultimate goal is to find the “efficient” spot on the axis in between fully automated and individually personalized mortgage modification solutions. The key is using all the data that’s out there to gain a better understanding of why individual borrowers are at risk of foreclosure, learning how better data can speed up the decision process for servicers evaluating modification options, and identifying common factors that could lead to the development and then deployment of more personalized foreclosure avoidance strategies.

The mortgage market has long been driven by quantitative analytics, but Joe Lonsdale and Rosco Hill framed a key question in their TEDx presentation that suggests a transformation of sorts is playing out at the nexus of Silicon Valley and Wall Street. In describing an exchange that a Palantir team had with a large bank that was evaluating new servicing-related technologies, a bank executive asked both his own IT-servicing department managers and the Palantir data mavens in the room whether the answers to today’s servicing-related challenges were more about finding mechanical or creative solutions? The answer is both, but it’s the underappreciated role of creativity in the development process of the data platform itself (i.e., turning an analytical tool into a decision-making platform) that gives Silicon Valley the edge in providing a real breakthrough on mortgage servicing.

PHOTO: Realtor and bank-owned signs displayed near a house for sale in Phoenix, Arizona, January 4, 2011. REUTERS/Joshua Lott