I understand points as entities with zero extension. (Is this correct?) Yet infinitely many points are said to compose space. It seems like even infinitely many zeros could never add up to a finite non-zero value. So, what's up with points? If they don't have any extension, what are they? As a follow up, does it make sense to think about points in space in a different way from how we think about points in time?

Yes, a point has length, depth, and height zero. So do two points, three points, and even as many points as there are natural numbers. But if you have as many points as there are real numbers (of which there are more than there are natural numbers), then that set of points may have some positive length, depth, or height, though it may not. (In that case, they will not have zero length, depth, and height but may have no assignable length, depth, or height.) The branch of mathematics in which such things are studied is called "measure theory". Exactly what a point is is another question. In mathematics, points may be regarded in a wide variety of ways, as is convenient. Are there any points in space itself? That's a disputed question, and an empirical one, not one on which philosophers can pronounce.