Is breathing in drywall dust bad for you?

My company has relocated to a brand new office. The problem is it is nowhere near done and they are making us work in the construction site. The whole place is a cloud of drywall dust and paint fumes. All the construction workers have masks but we don't. I've been working in it for 4 days now and my throat is sore and even now at Midnight, I feel like I have a 20 pound weight sitting on my chest and I feel like I am laboeing to breath. I had asthma as a kid - I wish I had my inhaler now...