Test if application is thread safe

Mateusz Gaweł :

I have simple application which simulates money transfer from one account to another. I want to write a test that will show that it is not thread safe.

There is a possibility that threads will go in such way that transfer will be done twice. Two threads scenario:

• acc1 = 1000$
• acc2 = 0$

• transfer 600$

  1. T1: get acc1 balance (1000)
  2. T2: get acc1 balance (1000)
  3. T1: subtract 600$ from account1 (400)
  4. T2: subtract 600$ from account2 (400)
  5. T1: increase acc2 by 600$ (600)
  6. T2: increase acc2 by 600$ (1200)

Currently my application doesn't support multithreading and it should fail, that is fine for me now. I am able to simulate error with debugger. However when I do thread test it's always succesful. I tried with different number of threads, sleeps, callabletasks

@Test
    public void testTransfer() throws AccountNotFoundException, NotEnoughMoneyException, InterruptedException {
        Callable<Boolean> callableTask = () -> {
            try {
                moneyTransferService.transferMoney(ACCOUNT_NO_1, ACCOUNT_NO_2, TRANSFER_AMOUNT);
                return true;
            } catch (AccountNotFoundException | NotEnoughMoneyException e) {
                e.printStackTrace();
                return false;
            }
        };

        List<Callable<Boolean>> callableTasks = new ArrayList<>();
        int transferTries = 2;
        for(int i = 0; i <= transferTries; i++) {
            callableTasks.add(callableTask);
        }

        ExecutorService executorService = Executors.newFixedThreadPool(2);
        executorService.invokeAll(callableTasks);

        Assert.assertEquals(ACCOUNT_BALANCE_1.subtract(TRANSFER_AMOUNT), accountRepository.getByAccountNumber(ACCOUNT_NO_1).get().getBalance());
        Assert.assertEquals(ACCOUNT_BALANCE_2.add(TRANSFER_AMOUNT), accountRepository.getByAccountNumber(ACCOUNT_NO_2).get().getBalance());
    }

Here's code for money transfer:

public void transferMoney(String accountFrom, String accountTo, BigDecimal amount) throws AccountNotFoundException, NotEnoughMoneyException {
        Account fromAccount = getAccountByNumber(accountFrom);
        Account toAccount = getAccountByNumber(accountTo);

        if (isBalanceSufficient(amount, fromAccount)) {

            //TODO this should be thread safe and transactional
            BigDecimal fromNewAmount = fromAccount.getBalance().subtract(amount);
            fromAccount.setBalance(fromNewAmount);

            // it's possible to fail junits with sleep but I dont want it in code obviously
//          Random random = new Random();
//          try {
//              Thread.sleep(random.nextInt(100));
//          } catch (InterruptedException e) {
//              // TODO Auto-generated catch block
//              e.printStackTrace();
//          }

            BigDecimal toNewAmount = toAccount.getBalance().add(amount);
            toAccount.setBalance(toNewAmount);

        } else {
            throw new NotEnoughMoneyException("Balance on account: " + fromAccount.getNumber() + " is not sufficient to transfer: " + amount);//TODO add currency
        }
    }

Persixty :

Welcome to the wonderful world of multi-threading. As a comment points out, it's going to be very hard to determine a way of proving anything without the full source code.

But also it can be hard to provoke threading errors. The first rule of multi-threading is you can't prove (or easily disprove) code is thread-safe by exercise (e.g. unit) testing.

Unsafe code may execute a billion times without error. On some platforms it may in fact be thread-safe but on others fail consistently. The idea that all Java code behaves the same on all platforms goes out of the window when you start using threads.

This code (invented content of your class) isn't guaranteed thread-safe in Java:

balance+=transaction;

But it's also such a small piece of code it potentially could be safe on some platforms or simply so fast it runs billions of times without error.

int temp=balance;
Thread.sleep(1000);
balance=temp+transaction;

Has a good chance of eventually failing on most platforms. But so what? It proves nothing about the original line of code and sometimes introducing delays masks issues particularly elsewhere.

The only way to verify or invalidate multi-threaded code is static analysis and a good knowledge of the language guarantees.

You can try running at high-load with (say) twice as many threads as your platform can actually run in parallel and making some guesses about the underlying code you've got a good chance of provoking a problem. But some errors may only occur at low load or any load inbetween.

Remember if you modify the code re-test and it works you've proved nothing. I'm not saying you shouldn't run such tests as a final check.

But never imagine unit-tests help demonstrate reliability for multi-threading in a way they help with single-threaded. That's particularly because different platforms may have different configurations (e.g. number of processes, cache levels, cores) and experience different levels of load.