Haze has struck me as one of the most real characters I have read in literature. I wouldn't doubt if every modern has had similar impulses against the world (though maybe not so rashly--O'Connor did enjoy the grotesque and exaggerated).
Despite her use of the grotesque, O'Connor makes Haze's reactions seem like his only realistic option. Haze feels physically estranged after separating from his father and escaping the war, and he feels spiritually estranged as evidenced in his obsessive denial of sin. He desires to see the world as a cohesive environment, but he knows he cannot fit into the world. Such is the consequence of sin.
Haze proves his own reaction against the world on page 69 when he encounters Sabbath, the preacher's daughter, "He looked at her irritably, for something in his mind was already contradicting him and saying that a bastard couldn't [be saved], that there was only one truth--that Jesus was a liar--and that her case was hopeless."
Haze fights violently in his mind to disprove Christ. He consistently assumes that the easiest way to avoid salvation is to deny Christ. Even after chapter nine of the novel, Haze tries to take seriously his own answer to his problems, and he becomes frustrated at Onnie Jay's theft and twisting of his new religion.
But why? Why does Haze become upset? The answer is apparent: he has built his own castle of "salvation" or solace by non-sin rather than redemption from sin, and it is threatened. However, Haze is not as keen at developing unique ideas as he would like people to believe. Haze is actually sensitive to the evil of others in the world. To prevent the same pain, Haze simply acts opposite of them. They acknowledge a god...there is none. There is uncleanliness...wrong...there is no standard.
When he runs out of ideas (after they are stolen), he reverts to a primitive, self-wounding lifestyle. He does not know how to remain clean and thus desires to destroy himself.
Edit: Maybe not destroy...maybe cleanse. Self-mortification. Cleansing by way of self-punishment, self-atonement.