New religion: robot grants a loan?

If AI makes credit decision we should know why it does so. Maybe we’ll be able to teach AI to explain such things to us. Still the question if it says true remains valid.

Tomasz Jurczak talks with me about AI, fintech, banks, and the future of humanity.

Original interview in Polish (2019): https://www.sztucznainteligencja.org.pl/robot-daje-kredyt-to-jak-nowa-religia-efni/

Tomasz Jurczak: why pediatrician, onkologist took a job in a bank?

Wojciech Bolanowski: The financials and workplace stability were the decisive factors. However, today there is nothing like stability of employment in banking; salaries are still fine, though.

But employees in banks are to be replaced with robots?

Indeed, there is a pressure to cut jobs in banking, both in Poland and globally. Various technologies replace some human workforce. This trend will grow for banking professionals are expensive. Sooner or later we see that more or less smart digital solutions take over our duties.

Credit analysis is a case?

Lending is a profitable business. Credit scoring made by robots is cheaper than human-driven and will increase returns from lending. Credit scoring, in line with back-office paperwork is one of the earliest, key implementation of AI to banking. This is not front-end touch-point, therefore banks are not afraid of losing their human touch.

Even today banks are obliged to inform customers about their reasons for credit denial. I wonder how they will explain if it is robot’s decision.

It’s super interesting challenge. We used software and algorithms to make the same what humans do, but much faster. So, we know what AI should do, but, in fact, we don’t know how it does so. We know how it learnt what it knows, because we teach AI. Finally, however, we don’t know how it works and it is serious problem.

If AI makes credit decision we should know why it does so. Maybe we’ll be able to teach AI to explain such things to us. Still the question if it says true remains valid. Additionally, will AI explain its reasoning to us the way we can get it?

AI shows us the data it used to calculate credit risk?

Explanation by data used is not clear and easy. Let’s imagine that AI invokes the data impacting credit decision and it is age, income, residency address and customer’s shoe size. Interesting to see how AI will justify this data set as meaningful. And still, above mentioned data is politically correct.

What if AI say you are not eligible to mortgage loan because you are a white male in mid-age. We have the right to ask “why?”

Why?

Because you are in the Board of a big bank. With women empowerment trend it is very likely you’ll be replaced by a woman of color. Therefore, we can’t grant you a mortgage.

On the other hand banks cannot deny their services based on race, color, religious belief (but they look at your age, though). If AI says there are important risk factors, what we will do?

The same with other professions. I am a journalist and AI can decide that freelancing is not stables, one can earn 10k monthly or just a tenth of that, or not a penny.

Sure, but it is profession related criterion and somehow acceptable by society. But if?AI decides women are at higher risk of losing a job and male journalist will be favored over females in credit decisions? We have a problem as it crosses the line and is not acceptable.

We say: the same work, the same job, the same salary to empower gender equality at work. What if AI proves that work males do is less important or valuable than females’? As consequence the AI managing a corportation will offer lower pay to males. Will we accept this, as a society?

There will be many problems of this kind. The worst will be when both AI and we, humans, are unable to explain how such decisions are made.

In this case we find ourselves believers of a new religion. We believe that all complex processes (like food supply, investment management, leading a bank, risk estimation etc.) are controlled by the mechanism (or artificial being?) who always knows better than we do; who does everything better than we do.

But there are errors, which AI can identify and correct …

Indeed, there are. However, the criteria of error and success are not clear anymore. If AI designs an airplane and the plane can’t flight – it is a clear mistake. What if AI manages capital investments? We don’t know what could happen if a human leads the same investment portfolio. Today we often argue which fund manager is actually better, right? When AI starts to beat the results of human managers should we always trust it? Even if we don’t know how AI works and whether the other AI would do better…

We can always regulate this …

Sure thing. But I have another example. AI says: I won’t give the loan to this company because it has too many women in its board. What we do in this case? Even if today risk analyst (human) thought that way (I doubt it would can happen) – he or she would never articulate such reason for credit denial. With AI it would be openly expressed. What regulators or courts should do? Should they overthrow AI’s decision and force banks?to accept increased risks in sake of gender equality? These are the question which should be answer in AI-driven future.

It all depends on our data …

The problem is not in data. It is about what we can learn about ourselves, what Facebook or Google already know. Browsing some websites can reveal we are pessimist or optimist. Let’s say we didn’t know this. How we react when we learn about ourselves such things?

For example I think I am very cost-cautious man but in 10-20 years a bank does not lend me some money. Banks decided I behave like very lavish man. I will ask for the reason and AI answers that I use to stroll through the park or linked on social media with some profiles or prefer watching pictures on Instagram with dull or cold colors. Maybe such behavior really matters, but I feel it would be unfair, incomprehensible to make credit decisions based on that.

Additionally data management is only as good as the data is. If our digital behavior does not constitute humans entirely – the data management will never be fully relevant.

The problem lies not only in wrong data, but also in data manipulation. Do you see such risk?

Regular algorithms can be manipulated, the “true” artificial intelligance generally can’t. Customer's credibility used to be assessed by rather simple algorithms. In the past some customers manipulated their credibility swapping the very same money between different banks. AI wouldn’t be misled that way.I have no doubt – in the future we will be unable to cheat on AI. It will be really smart, digital mind, not a regular software product.

Polish banks are considered technologically advanced. However, Poland is not an important fintech hub.

Poland just could not win in the competition for recognized “fintech hub” – this positioning is already occupied by such strong players like e.g. London. Although in Poland most banks run technologies and provide solutions which are perceived as a top notch fintech innovations in the UK.

How does it look like in Asia? Are they as innovative as Polish banks and financial institutions?

Middle East banking sector is historically well-connected with Anglo-Saxon financial world. They take a lot after Great Britain. For example in Bahrain we have the open banking regulation, very similar to PSD2, there is a huge appetite for fintech ecosystem development.

And what about?Singapore?

Singapore is quite different, just like other Southeast Asia. In Poland, payments go disruptive, but in context of payment cards. The cards went virtual, got tokenized, were put into smartphones and wearables. In Asia there was no huge card market and payment innovations are different. There is AliPay, WeChat, QR codes, face recognition and biometrics in Asia financial services landscape. Banks in Southeast Asia are very advanced regarding digital technology and fintech have the same challenge like in Poland – they offer solutions which are not substantially better than those in banks.

On the other hand UK was an undeveloped market, regarding retail banking services and technologies. When fintech came they brough disruptive change, quantum leap. Tech savvy customers went to fintech neobanks. In Poland they got those solutions in regular banks.

How has Polish banking sector evolved for last ten years?

It changed a lot in Poland, new leaders of financial innovation emerged. I watch carefully PKO BP, where I used to work in 2010-2015 and see a neverending progress. The bank is market leader by volumes, but it’s gradually becoming the innovation leader, too. It shapes the market by having millions of customers who adopt new solutions. It became another challenge for fintech in Poland.

What innovations will dominate banking in next few years?

In my view push for innovation in banking will weaken over time. This will be long-term trend. In short-term I see biometrics as a driver for innovation. Personally, I am not a fan. Biometrics limits freedom and privacy, especially when it becomes more vulnerable to hacking. Today, if a regular password is hacked, the change of it is a sufficient countermeasure. What if your retina or fingerprint gets hacked? Despite of this I think that in five years biometrics becomes the major way of customer authentication for payments and other services.

Will banks as we know extinct? When it will happen? Will financial services survive?

Describing future financial world is very hard, even for the next twenty years. We can imagine expansion of the welfare state trend of guaranteed services so successful that people will not need to work a lot nor earn much money to live satisfying lives.

I think your vision is a fairytale, not a realistic scenario.

Why do you think so? Economy expansion and GDP growth directly generates harmful climate changes. Therefore logging, mining, burning fossil fuel will become out of fashion and people will limited such activities. It is rational reaction.

Loan, the pacemaker of capitalism, will lose its importance; we have already seen some signs of it. The trend of rental instead of buying is re-shaping realty markets. Soon, when we start to rent TV sets and kitchenware, the consumer loan will lose its significance. In the future, a state can provide free education, healthcare, social care, music concerts, transportation (like trains, buses, and flights) and holidays trips. If this happens, we won’t need much money, savings, investments and loans … Generally, we can imagine that every financial service we need today will become useless in few decades.

When climate challenge and lack of fossil fuel make airplane flights obsolete?our children can always visit Croatia or exotic places in virtual reality room. They will not perceive lack of airplanes as limitation or discomfort. Similarly the right to beef consumption. If today we say that animal husbandry deteriorates global climate (and beef is the worst) beef will gradually become expensive, its production limited and then, probably, ceased. We will not feel that we compromised our quality of life.

With relevant change of human mentality …

I assume that humans don’t want possesion for the sake of itself. They need things to use them and the trend will grow. In my generation, where ownership was something bad under communism, we starve to buy a house, a car for ourselves. But it will change. Rememeber young generation. Look at smartphone: why we should buy it or lease it (the mobile plan is a kind of customer leasing) or take a loan to purchase new iPhone? It is enough if we pay for every minute we actually use the device. Let’s imagine that the state is providing the right to use a smartphone for every citizen – and nobody will have to buy mobile again. I just don’t know who will cover the final cost of it.

Aren’t you afraid that the state or a big tech, whoever will provide those goods to citizens, will know everything about them? Today we still enjoy some oases of freedom and privacy because not all our data are connected.

And there is still cash over there. But we can easily imagine that every note will have RFID marker within which allows continuous cash-tracking. We will know in whose wallet the note is and, using public cameras and face recognition technology we will recognize the person who pay with the note.

Freedom and privacy are not supreme values, humans can give them away in exchange. We can trade of our privacy for convenience, security, and for solving our everyday problems. It is a fair price, isn’t it?