There are lots of ISV's out there within the SharePoint ecosystem and unfortunately a lot of them do not follow the standard practices for deployment, often because of how they wish to handle licensing of their products. This means that the use of WSP packages is often bypassed for direct installation executable.
There are many risks of using 3rd Party Products and it is recommended that a full review of the product be executed.
The below guidelines will evolve over time to help the community to execute these reviews.
Security
- Is Code Access Security applied to the product to minimise footprint of components the product has access to?
- Does it require any "abnormal" security requirements e.g. administrator access?
Performance
- Are there any memory leaks with this product when it is load tested?
Scalability
- Does the product support multiple server SharePoint farms?
Supportability
- Has it been built in such a way it will handle service pack and cumulative updates?
- Is it SharePoint 2010 ready?
- Are there any known conflicts with other SharePoint ISV products?
