The `latest` build remains as is, but all others are modified to:

* Use regular Python instead of conda. `pip install` is much faster
  than conda, but scipy isn't available. Numpy is still tested.

* Compile in debug mode instead of release.

* Skip CMake build tests. For some reason, CMake configuration is very
  slow on AppVeyor and these tests are almost entirely CMake.

The changes reduce build time to about 1/3 of the original. The `latest` 
config still covers scipy, release mode and the CMake build tests, so 
the others don't need to.