The problem is that because the curvature of the outside of the balls is different to the curvature of the cone, the balls will only rest against a very small part of the cone. This will put massive pressure on them, and they'll fail. Probably quite quickly. Ball and cone bearings are machined to very high tolerances, which is why something so small is able to take such large loads. They problem wouldn't be the fact it'd wear quickly - the problem is that it would fail, probably fairly catastrophically.
The problem is that because the curvature of the outside of the balls is different to the curvature of the cone, the balls will only rest against a very small part of the cone. This will put massive pressure on them, and they'll fail. Probably quite quickly. Ball and cone bearings are machined to very high tolerances, which is why something so small is able to take such large loads. They problem wouldn't be the fact it'd wear quickly - the problem is that it would fail, probably fairly catastrophically.