I think I've figured it out. Checking that ContentVersion is creatable alone is not sufficient.
I also needed to check that the current user (as indicated by getUserInfo()) has the field UserPermissionsSFContentUser and that it is true.
Indicates whether the user is allocated one Salesforce CRM Content User License (true) or not (false). Label is Salesforce CRM User. The Salesforce CRM Content User license grants the user access to the Salesforce CRM Content application.