Here's why it makes sense:
* Chalk is a white, powdery substance.
* Cheese is a solid, often yellowish food.
The two are completely different in appearance, texture, and purpose.
The idiom emphasizes the stark contrast between two things. It's used to highlight how dissimilar or incompatible two things are.
Example:
"Their personalities are as different as chalk from cheese. One is quiet and introverted, while the other is loud and outgoing."