I am pretty sure you are
My beginning is the beginning of glass
Glass begins as SAND
My middle gives you a hand take one,
a slang patriarch, and early college credit.
"A hand take one" might be a PAW with the last letter taken away, for PA.
A slang patriarch is also a PA, and you get early college credit via AP courses.
Together these give PAP
OP indicates the intent here was literally HAND with the first letter taken away, for AND.
That, with the PAP clearly clued by the "patriarch" line, gives a "middle" of ANDPAP.
That's not a word, and the usual expectation is that the pieces are words.
So I reject that and go with the interpretation I put above. :)
I end with a downtrodden stone.
PAPER beats (a thus downtrodden) stone in roshambo (rock-paper-scissors). (thanks @North)
The title ...
SANDPAPER is, of course, rough.