This is a fundamental misunderstanding/misgiving. It's NOT matrix algebra. It's a neural network approach. It is correct that we can't understand the inner workings of the neural network any better than we can understand how a proper biological brain works - which means we can in principle understand it, but the degree of complexity stands in the way of a practically feasible approach of understanding it.
The key difference between matrix algebra and a neural network is that the former is deterministic and pre-determined; the latter by contrast exhibits emergent behavior. If you look at it from a distance and treat both a matrix algebraic algorithm (let's say something like a Markov chain) and a neural network as a black box, they might seem somewhat similar in that they can exhibit similar behavior. However, the inner workings, the way they're 'built'/designed and their capabilities are fundamentally different.
What this misunderstanding illustrates is how deeply rooted the misconceptions surrounding 'AI' are (understandably!) and how difficult this makes it for the present population to intuitively grasp the capabilities of this form of AI. I expect that this will change over time just like the present generations have managed to come to grips with digital technology, the internet etc - all things that the vast majority of people don't really understand thoroughly either, but most of us are fairly well aware of the possibilities and impossibilities associated with these technologies. In a similar vein, I think we'll also come to grips with practical applications of neural networks, and in the near future, quantum computing.