For the given premises P and Not P, is P a valid derivation? Shouldn't the derivation be true for all the premises for it to be valid or is it not sound and yet valid? But aren't we determining its unsoundness by virtue of something other than the content of those premises?
Given premises P and not P, it is indeed valid to derive P. I don't know of any logical systems, including non-classical systems, that would deny the validity of that derivation. (A valid derivation needn't use all of its premises: "P; Q; therefore, P" is valid.) The derivation you gave isn't sound, however, because not all of the premises are true: it's guaranteed that one of the premises is false (even if we don't know which one). Yes, we're ascertaining the unsoundness of the derivation without knowing the content of its premises, but that's perfectly fine: If you know that the form of the derivation guarantees that it has a false premise, you don't need to know anything more in order to know that the derivation is unsound.