The biggest clue here is
the professor's name, which we should read phonetically as "max plus".
This suggests that we interpret the "nebulous hints" as
indicating that we perform our "matrix multiplication" replacing addition with max() and multiplication with __+__. (We'll take explicit negation as meaning what it says, though, because the max operation doesn't admit inverses.)
So our five equations are
max(a+3,b+3,c-2,d-2,e) = 3
max(a-2,b+1,c+1,d+1,e-2) = 1
max(a-1,b-1,c,d+2,e+3) = 4
max(a,b+1,c-2,d,e-1) = 1
max(a+1,b+3,c+3,d+1,e+4) = 5.
I'm not sure whether there's a standard systematic way to solve these, but let's see what we can do ad hoc. First,
consider the fourth and fifth equations. The five arguments to max() have been increased by 1,2,5,1,5 respectively, and the result has increased by 4. It follows that (1) the maxes aren't in the same place both times (else the increase in result would have been 1, 2, or 5), (2) the max in the fourth equation isn't c-2 or e-1 (else the increase in result would have been at least 5), (3) the max in the fifth equation is c+3 or e+4 (else the increase in result would have been at most 2). So either c=2 or e=1. The former is impossible because of the second equation. So e=1.
our equations become, after a little simplification,
max(a+3,b+3,c-2,d-2) = 3
max(a-2,b+1,c+1,d+1) = 1
max(a-1,b-1,c,d+2,4) = 4 [i.e., a<=5, b<=5, c<=4, d<=2]
max(a,b+1,c-2,d) = 1
max(a+1,b+3,c+3,d+1,5) = 5 [i.e., a<=4, b<=2, c<=2, d<=4].
So those two "already saturated" equations tell us that a<=4, b<=2, c<=2, d<=2. The first equation strengthens this a little: a<=0, b<=0. Since the unknowns are single digits, this in fact implies a=b=0.
So now we have
a=0, b=0, e=1; c<=2, d<=2
max(3,c-2,d-2) = 3 [i.e., c<=5, d<=5]
max(-1,c+1,d+1) = 1 [i.e., c<=0, d<=0]
max(1,c-2,d) = 1 [i.e., c<=3, d<=1]
from which we find
that our solution is a=0, b=0, c=0, d=0, e=1. (It's easy to verify that this does work.)
So it seems that the secret combination may just be