If that is carpet in his cage I would get it out. Ingesting carpet fibers can lead to a digestive blockage occurring, so if there is any possibility he is ingesting the carpet fibers, you need to prevent him having access to carpet.
Other options are lining the cage with fleece fabric which is less likely to have fibers chewed and swallowed, or sheets of newspaper(as long as he's not trying to consume this as well). He may dig and shred it, but that is ok as it is a good enrichment activity and is easily cleaned up and replaced.