Fairness and Equality, Libertarian-style

I recently had a vigorous, healthy (i.e. mean and nasty) debate in a Libertarian discussion group on social media. I argued that rich children have an unfair advantage over poor children because poor children are trapped in failing public schools, and rich children can attend good private schools. The two people I was debating (one of whom was nice but foolish, the other absolutely vicious) could not think past the mistaken idea that I was saying it is unfair because the rich have more money and can afford to send their children to private schools, and poor people can't, which is NOT my argument. My argument was the libertarian argument that the government uses VIOLENT FORCE to compel poor children to attend the failing public schools, because of our rotten socialized k-12 public education system and compulsory attendance. Because the government uses this force against poor children, but not rich children, it is UNFAIR for the poor children, and the law is treating rich children differently from poor children, and giving them preferential treatment. I was making a libertarian argument against government and for the privatizing of primary school, yet the dim, dull people I argued with somehow could not get it around their brains that I was not making a Marxist/Socialist argument to outlaw private schools. Very frustrating, but, obviously, political dialogue is a good and vital element of a functioning democracy.