Notes on setting up the windows development environment ======================================================== Overview ---------- calibre and all its dependencies are compiled using Visual Studio 2008 express edition (free from MS). All the following instructions must be run in a visual studio command prompt unless otherwise noted. calibre contains build script to automate the building of the calibre installer. These scripts make certain assumptions about where dependencies are installed. Your best best is to setup a VM and replicate the paths mentioned below exactly. Basic dependencies -------------------- Install cygwin and setup sshd (optional). Used to enable automation of the calibre build VM from linux, not needed if you are building manually. Install MS Visual Studio 2008, cmake, python and WiX. Set CMAKE_PREFIX_PATH environment variable to C:\cygwin\home\kovid\sw This is where all dependencies will be installed. Add C:\Python27\Scripts and C:\Python27 to PATH Edit mimetypes.py in C:\Python27\Lib and set _winreg = None to prevent reading of mimetypes from the windows registry Install setuptools from http://pypi.python.org/pypi/setuptools If there are no windows binaries already compiled for the version of python you are using then download the source and run the following command in the folder where the source has been unpacked:: python setup.py install Run the following command to install python dependencies:: easy_install --always-unzip -U ipython mechanize pyreadline python-dateutil dnspython cssutils clientform pycrypto Install BeautifulSoup 3.0.x manually into site-packages (3.1.x parses broken HTML very poorly) Install pywin32 and edit win32com\__init__.py setting _frozen = True and __gen_path__ to a temp dir (otherwise it tries to set it to a dir in the install tree which leads to permission errors) Note that you should use:: import tempfile __gen_path__ = os.path.join( tempfile.gettempdir(), "gen_py", "%d.%d" % (sys.version_info[0], sys.version_info[1])) Use gettempdir instead of the win32 api method as gettempdir returns a temp dir that is guaranteed to actually work. Also edit win32com\client\gencache.py and change the except IOError on line 57 to catch all exceptions. SQLite --------- Put sqlite3*.h from the sqlite windows amlgamation in ~/sw/include APSW ----- Download source from http://code.google.com/p/apsw/downloads/list and run in visual studio prompt python setup.py fetch --all build --missing-checksum-ok --enable-all-extensions install test OpenSSL -------- First install ActiveState Perl if you dont already have perl in windows Download and untar the openssl tarball, follow the instructions in INSTALL.W32 (use no-asm) to install use prefix q:\openssl perl Configure VC-WIN32 no-asm enable-static-engine --prefix=Q:/openssl ms\do_ms.bat nmake -f ms\ntdll.mak nmake -f ms\ntdll.mak test nmake -f ms\ntdll.mak install Qt -------- Extract Qt sourcecode to C:\Qt\4.x.x. Qt uses its own routine to locate and load "system libraries" including the openssl libraries needed for "Get Books". This means that we have to apply the following patch to have Qt load the openssl libraries bundled with calibre: --- src/corelib/plugin/qsystemlibrary.cpp 2011-02-22 05:04:00.000000000 -0700 +++ src/corelib/plugin/qsystemlibrary.cpp 2011-04-25 20:53:13.635247466 -0600 @@ -110,7 +110,7 @@ HINSTANCE QSystemLibrary::load(const wch #if !defined(QT_BOOTSTRAPPED) if (!onlySystemDirectory) - searchOrder << QFileInfo(qAppFileName()).path(); + searchOrder << (QFileInfo(qAppFileName()).path().replace(QLatin1Char('/'), QLatin1Char('\\')) + QString::fromLatin1("\\DLLs\\")); #endif searchOrder << qSystemDirectory(); Now, run configure and make:: -no-plugin-manifests is needed so that loading the plugins does not fail looking for the CRT assembly configure -opensource -release -qt-zlib -qt-gif -qt-libmng -qt-libpng -qt-libtiff -qt-libjpeg -release -platform win32-msvc2008 -no-qt3support -webkit -xmlpatterns -no-phonon -no-style-plastique -no-style-cleanlooks -no-style-motif -no-style-cde -no-declarative -no-scripttools -no-audio-backend -no-multimedia -no-dbus -no-openvg -no-opengl -no-qt3support -confirm-license -nomake examples -nomake demos -nomake docs -no-plugin-manifests -openssl -I Q:\openssl\include -L Q:\openssl\lib && nmake SIP ----- Available from: http://www.riverbankcomputing.co.uk/software/sip/download :: python configure.py -p win32-msvc2008 nmake nmake install PyQt4 ---------- Compiling instructions:: python configure.py -c -j5 -e QtCore -e QtGui -e QtSvg -e QtNetwork -e QtWebKit -e QtXmlPatterns --verbose nmake nmake install Python Imaging Library ------------------------ Install as normal using installer at http://www.lfd.uci.edu/~gohlke/pythonlibs/ Test it on the target system with calibre-debug -c "import _imaging, _imagingmath, _imagingft, _imagingcms" ICU ------- Download the win32 msvc9 binary from http://www.icu-project.org/download/4.4.html Note that 4.4 is the last version of ICU that can be compiled (is precompiled) with msvc9 Put the dlls into sw/bin and the unicode dir into sw/include and the contents of lib int sw/lib Libunrar ---------- http://www.rarlab.com/rar/UnRARDLL.exe install and add C:\Program Files\UnrarDLL to PATH lxml ------ http://pypi.python.org/pypi/lxml jpeg-7 ------- Copy:: jconfig.vc to jconfig.h, makejsln.vc9 to jpeg.sln, makeasln.vc9 to apps.sln, makejvcp.vc9 to jpeg.vcproj, makecvcp.vc9 to cjpeg.vcproj, makedvcp.vc9 to djpeg.vcproj, maketvcp.vc9 to jpegtran.vcproj, makervcp.vc9 to rdjpgcom.vcproj, and makewvcp.vc9 to wrjpgcom.vcproj. (Note that the renaming is critical!) Load jpeg.sln in Visual Studio Goto Project->Properties->General Properties and change Configuration Type to dll Add #define USE_WINDOWS_MESSAGEBOX to jconfig.h (this will cause error messages to show up in a box) Change the definitions of GLOBAL and EXTERN in jmorecfg.h to #define GLOBAL(type) __declspec(dllexport) type #define EXTERN(type) extern __declspec(dllexport) type cp build/jpeg-7/Release/jpeg.dll bin/ cp build/jpeg-7/Release/jpeg.lib build/jpeg-7/Release/jpeg.exp cp build/jpeg-7/jerror.h build/jpeg-7/jpeglib.h build/jpeg-7/jconfig.h build/jpeg-7/jmorecfg.h include/ zlib ------ nmake -f win32/Makefile.msc nmake -f win32/Makefile.msc test cp zlib1.dll* ../../bin cp zlib.lib zdll.* ../../lib cp zconf.h zlib.h ../../include libpng --------- cp scripts/CMakelists.txt . mkdir build Run cmake-gui.exe with source directory . and build directory build You will have to point to sw/lib/zdll.lib and sw/include for zlib Also disable PNG_NO_STDIO and PNG_NO_CONSOLE_IO Now open PNG.sln in VS2008 Set Build type to Release cp build/libpng-1.2.40/build/Release/libpng12.dll bin/ cp build/libpng-1.2.40/build/Release/png12.* lib/ cp build/libpng-1.2.40/png.h build/libpng-1.2.40/pngconf.h include/ freetype ----------- Edit *all copies* of the file ftoption.h and add to generate a .lib and a correct dll #define FT_EXPORT(return_type) __declspec(dllexport) return_type #define FT_EXPORT_DEF(return_type) __declspec(dllexport) return_type VS 2008 .sln file is present, open it Change active build type to release mutithreaded Project->Properties->Configuration Properties change configuration type to dll cp build/freetype-2.3.9/objs/release_mt/freetype.dll bin/ Now change configuration back to static for .lib cp build/freetype-2.3.9/objs/win32/vc2008/freetype239MT.lib lib/ cp -rf build/freetype-2.3.9/include/* include/ expat -------- Has a VC 6 project file expat.dsw Set active build to Relase and change build type to dll cp build/expat-2.0.1/win32/bin/Release/*.lib lib/ cp build/expat-2.0.1/win32/bin/Release/*.exp lib/ cp build/expat-2.0.1/win32/bin/Release/*.dll bin/ cp build/expat-2.0.1/lib/expat.h build/expat-2.0.1/lib/expat_external.h include/ libxml2 ------------- cd win32 cscript configure.js include=C:\cygwin\home\kovid\sw\include lib=C:\cygwin\home\sw\lib prefix=C:\cygwin\home\kovid\sw zlib=yes iconv=no nmake /f Makefile.msvc nmake /f Makefile.msvc install mv lib/libxml2.dll bin/ cp ./build/libxml2-2.7.5/win32/bin.msvc/*.manifest bin/ kdewin32-msvc ---------------- Get it from http://www.winkde.org/pub/kde/ports/win32/repository/kdesupport/ mkdir build Run cmake Set build type to release and configuration to dll Build cp build/kdewin32-msvc-0.3.9/build/include/* include/ cp build/kdewin32-msvc-0.3.9/build/bin/Release/*.dll bin/ cp build/kdewin32-msvc-0.3.9/build/bin/Release/*.lib lib/ cp build/kdewin32-msvc-0.3.9/build/bin/Release/*.exp lib/ cp -r build/kdewin32-msvc-0.3.9/include/msvc/ include/ cp build/kdewin32-msvc-0.3.9/include/*.h include/ fontconfig --------------- Get it from http://www.winkde.org/pub/kde/ports/win32/repository/win32libs/ mkdir build Remove subdirectory test from the bottom of CMakeLists.txt run cmake Set build type to release and project config to dll Right click on the fontconfig project and select properties. Add sw/include/msvc to the include paths Build only fontconfig cp build/fontconfig-msvc-2.4.2-3/build/src/Release/*.dll bin cp build/fontconfig-msvc-2.4.2-3/build/src/Release/*.lib lib cp build/fontconfig-msvc-2.4.2-3/build/src/Release/*.exp lib cp -r build/fontconfig-msvc-2.4.2-3/fontconfig/ include/ Also install the etc files from the font-config-bin archive from kde win32libs It contains correct fonts.conf etc. poppler ------------- In Cmake: disable GTK, Qt, OPenjpeg, cpp, lcms, gtk_tests, qt_tests. Enable qt4, jpeg, png and zlib NOTE: poppler must be built as a static library, unless you build the qt4 bindings Now do the same for the pdftohtml project cp poppler/*.h ~/sw/include/poppler && cp goo/*.h ~/sw/include/poppler/goo && cp splash/*.h ~/sw/include/poppler/splash && cp build/Release/poppler.lib ../../lib/ && cp build/utils/Release/*.exe ../../bin/ podofo ---------- Add the following three lines near the top of CMakeLists.txt SET(WANT_LIB64 FALSE) SET(PODOFO_BUILD_SHARED TRUE) SET(PODOFO_BUILD_STATIC FALSE) cp build/podofo-*/build/src/Release/podofo.dll bin/ cp build/podofo-*/build/src/Release/podofo.lib lib/ cp build/podofo-*/build/src/Release/podofo.exp lib/ cp build/podofo-*/build/podofo_config.h include/podofo/ cp -r build/podofo-*/src/* include/podofo/ You have to use >=0.8.2 The following patch (against -r1269) was required to get it to compile: Index: src/PdfFiltersPrivate.cpp =================================================================== --- src/PdfFiltersPrivate.cpp (revision 1261) +++ src/PdfFiltersPrivate.cpp (working copy) @@ -1019,7 +1019,7 @@ /* * Prepare for input from a memory buffer. */ -GLOBAL(void) +void jpeg_memory_src (j_decompress_ptr cinfo, const JOCTET * buffer, size_t bufsize) { my_src_ptr src; ImageMagick -------------- Edit VisualMagick/configure/configure.cpp to set int projectType = MULTITHREADEDDLL; Run configure.bat in a visual studio command prompt Run configure.exe generated by configure.bat Edit magick/magick-config.h Undefine ProvideDllMain and MAGICKCORE_X11_DELEGATE Now open VisualMagick/VisualDynamicMT.sln set to Release Remove the CORE_xlib and UTIL_Imdisplay project CORE_Magick++ calibre --------- Take a linux calibre tree on which you have run the following command:: python setup.py stage1 and copy it to windows. Run:: python setup.py build python setup.py win32_freeze This will create the .msi in the dist directory.