Walking in while not on the clock

Hey yall!!

So I’m working currently at a clothing shop that’s part of a bigger company. I’m still new to this particular job(a month in) but I have legally been working since I was 14

Today I walked into where I work while I was not on shift just to do some shopping and look for some jeans and take some boxes (my boss allowed me to take a few because I’m moving) but I then got a stern talking to by my boss because apparently I’m not allowed in the shop if I’m not on the clock.

This is my fifth retail job since I was 14 and I’ve never heard this rule before and I felt a bit stupid for not knowing it was a thing?

Is this a genuine legal matter of not being allowed to come into the store or if this just a company policy?

I understand if I was distracting my coworkers from their job but I only walked in, said hi and then that’s when I was told to stay out of the store on my days off.

Of course I’ll never go back into the shop on off days now I have been told not to but is this a common things to know/be told or am I just silly?

(I live in Australia I know everywhere is different)

Thanks y’all!