Okay so when I roll down the windows(at least the driver side one), I can hear that something's scratching it when it rolls down, is there any easy way to clean it from the inside somehow? 
Car is Octavia MKII 2011 if that's important.

It’s most likely grit and dirt caught in the weather strip on the outside. Put the window down and use a cloth with some all purpose cleaner or shampoo from your wash bucket to clean it along the entire width of the channel (from mirror to pillar). 
I do this every week when cleaning the car, but I’m slightly weird so that might be overkill. 

On 20/01/2021 at 11:31, UndertheRadar said:

It’s most likely grit and dirt caught in the weather strip on the outside. Put the window down and use a cloth with some all purpose cleaner or shampoo from your wash bucket to clean it along the entire width of the channel (from mirror to pillar). 
I do this every week when cleaning the car, but I’m slightly weird so that might be overkill. 

Definitely not going to do it every week haha, but will try, thanks! I recently bought the car, dirt in there seems to scratch the window a little at this point, so will try to clean it asap!

Yes. I’ve been caught out a few times, where I’ve been in terrible road conditions and having to lower the window at a tool booth on the motorway, and it’s trapped a bit of grit and marked the glass. If it’s reasonably light, you can get most of it out with a glass polishing pad and a glass cutting polish. 
you could also use a horsehair detailing brush and some all purpose cleaner, and just go around them from the outside when washing the rest of the car, to avoid any buildup getting trailed down into the channel when the windows going down.