sync with Kovid's branch

Changelog.yaml

@@ -19,6 +19,55 @@
 #   new recipes:
 #     - title: 
 
+- version: 0.9.3
+  date: 2012-10-19
+
+  new features:
+    - title: "Conversion: Add support for CSS 3 selectors by switching to the new cssselect module"
+
+    - title: "Drivers for the WayteQ xBook and the Topwise Android tablet"
+      tickets: [1066083,1067105]
+
+    - title: "Add an output profile for the Kindle PaperWhite"
+
+    - title: "Kobo driver: Improve performance when a large number of shelves are created on the device."
+      tickets: [1066633]
+
+    - title: "Catalogs: Put catalog sections at top level of ToC for EPUB/AZW3 catalogs."
+
+    - title: "Adding books: When duplicates are found, show the list of possible duplicates in calibre with title and author."
+ 
+  bug fixes:
+    - title: "KF8 Input: Fix conversion/viewing of KF8 files generated with the unreleased? kindlegen 2.7."
+      tickets: [1067310]
+
+    - title: "Kindle driver: Increase the size of the cover thumbnails sent by calibre to the device. This fixes the problem of cover thumbnails not showing up on the PaperWhite"
+
+    - title: "MTP driver: Fix sorting on the title column of the device view."
+      tickets: [1067562]
+
+    - title: "Catalogs: Fix regression that caused authors and titles to be be incorrectly listed under symbols on OSX < 10.8."
+
+    - title: "Catalogs: Fix error when generating catalog in non English locale and the user has specified a prefix rule using a Yes/no column."
+      tickets: [1065452]
+
+    - title: "E-book viewer: Remove the reload entry from the context menu as it is not supported in paged mode."
+      tickets: [1065615]
+
+  improved recipes:
+    - Richmond Times Dispatch
+    - Various Polish news sources
+    - Aksiyon dergisi
+    - Spektrum der Wissenschaft
+    - Zeit Online
+    - Baltimore Sun
+    - LWN Weekly
+    - The Sun
+
+  new recipes:
+    - title: Various Polish news sources
+      author: fenuks
+
 - version: 0.9.2
   date: 2012-10-11
 

manual/faq.rst

@@ -69,8 +69,8 @@ If you have a hand edited TOC in the input document, you can use the TOC detecti
 
 Finally, I encourage you to ditch the content TOC and only have a metadata TOC in your ebooks. Metadata TOCs will give the people reading your ebooks a much superior navigation experience (except on the Kindle, where they are essentially the same as a content TOC).
 
-The covers for my MOBI files have stopped showing up in Kindle for PC/Kindle for Android/etc.
+The covers for my MOBI files have stopped showing up in Kindle for PC/Kindle for Android/iPad etc.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 This is caused by a bug in the Amazon software. You can work around it by going
 to Preferences->Output Options->MOBI output and setting the "Enable sharing
@@ -284,8 +284,8 @@ Use the 'Connect to iTunes' method in the 'Getting started' instructions in `Cal
 
 This method only works on Windows XP and higher, and OS X 10.5 and higher. Linux is not supported (iTunes is not available in linux) and OS X 10.4 is not supported.
 
-How do I use |app| with my Android phone/tablet?
+How do I use |app| with my Android phone/tablet or Kindle Fire HD?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 There are two ways that you can connect your Android device to calibre. Using a USB cable -- or wirelessly, over the air.
 The first step to using an Android device is installing an ebook reading
@@ -435,7 +435,7 @@ any |app| developers will ever feel motivated enough to support it. There is how
 that allows you to create collections on your Kindle from the |app| metadata. It is available
 `from here <http://www.mobileread.com/forums/showthread.php?t=118635>`_. 
 
-.. note:: Amazon have removed the ability to manipulate collections completely in their newer models, like the Kindle Touch and Kindle Fire, making even the above plugin useless. If you really want the ability to manage collections on your Kindle via a USB connection, we encourage you to complain to Amazon about it, or get a reader where this is supported, like the SONY Readers.
+.. note:: Amazon have removed the ability to manipulate collections completely in their newer models, like the Kindle Touch and Kindle Fire, making even the above plugin useless. If you really want the ability to manage collections on your Kindle via a USB connection, we encourage you to complain to Amazon about it, or get a reader where this is supported, like the SONY or Kobo Readers.
 
 I am getting an error when I try to use |app| with my Kobo Touch?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

recipes/adventure_zone_pl.recipe

@@ -3,7 +3,7 @@ import re
 class Adventure_zone(BasicNewsRecipe):
     title          = u'Adventure Zone'
     __author__        = 'fenuks'
-    description   = 'Adventure zone - adventure games from A to Z'
+    description   = u'Adventure zone - adventure games from A to Z'
     category       = 'games'
     language       = 'pl'
     no_stylesheets = True
@@ -11,7 +11,9 @@ class Adventure_zone(BasicNewsRecipe):
     max_articles_per_feed = 100
     index='http://www.adventure-zone.info/fusion/'
     use_embedded_content=False
-    preprocess_regexps     = [(re.compile(r"<td class='capmain'>Komentarze</td>", re.IGNORECASE), lambda m: '')]
+    preprocess_regexps     = [(re.compile(r"<td class='capmain'>Komentarze</td>", re.IGNORECASE), lambda m: ''),
+    (re.compile(r'\<table .*?\>'), lambda match: ''),
+    (re.compile(r'\<tbody\>'), lambda match: '')]
     remove_tags_before= dict(name='td', attrs={'class':'main-bg'})
     remove_tags= [dict(name='img', attrs={'alt':'Drukuj'})]
     remove_tags_after= dict(id='comments')
@@ -52,6 +54,11 @@ class Adventure_zone(BasicNewsRecipe):
 
     def preprocess_html(self, soup):
         footer=soup.find(attrs={'class':'news-footer middle-border'})
+        r = soup.find(name='td', attrs={'class':'capmain'})
+        if r:
+            r.name='h1'
+        for item in soup.findAll(name=['tr', 'td']):
+            item.name='div'
         if footer and len(footer('a'))>=2:
             footer('a')[1].extract()
         for item in soup.findAll(style=True):

recipes/aksiyon_derigisi.recipe

@@ -17,18 +17,13 @@ class Aksiyon (BasicNewsRecipe):
     category               = 'news, haberler,TR,gazete'
     language               = 'tr'
     publication_type = 'magazine'
-    #extra_css              = ' body{ font-family: Verdana,Helvetica,Arial,sans-serif } .introduction{font-weight: bold} .story-feature{display: block; padding: 0; border: 1px solid; width: 40%; font-size: small} .story-feature h2{text-align: center; text-transform: uppercase} '
+    auto_cleanup = True
-   #keep_only_tags    = [dict(name='font', attrs={'class':['newsDetail','agenda2NewsSpot']}),dict(name='span', attrs={'class':['agenda2Title']}),dict(name='div', attrs={'id':['gallery']})]
-    remove_tags  = [dict(name='img', attrs={'src':[ 'http://medya.aksiyon.com.tr/aksiyon/images/logo/logo.bmp','/aksiyon/images/template/green/baslik0.gif','mobile/home.jpg']}) ]
-
     cover_img_url = 'http://www.aksiyon.com.tr/aksiyon/images/aksiyon/top-page/aksiyon_top_r2_c1.jpg'
     masthead_url = 'http://aksiyon.com.tr/aksiyon/images/aksiyon/top-page/aksiyon_top_r2_c1.jpg'
     remove_empty_feeds= True
-    remove_attributes = ['width','height']
-
     feeds          = [
+                      ( u'KAPAK', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=26'),
                       ( u'ANASAYFA', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=0'),
-                      ( u'KARAKUTU', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=11'),
                       ( u'EKONOMİ', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=35'),
                       ( u'EKOANALİZ', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=284'),
                       ( u'YAZARLAR', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=17'),
@@ -37,17 +32,15 @@ class Aksiyon (BasicNewsRecipe):
                       ( u'ARKA PENCERE', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=27'),
                       ( u'DÜNYA', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=32'),
                       ( u'DOSYALAR', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=34'),
+                      ( u'KARAKUTU', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=11'),
                       ( u'KÜLTÜR & SANAT', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=12'),
-                      ( u'KAPAK', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=26'),
                       ( u'SPOR', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=38'),
                       ( u'BİLİŞİM - TEKNOLOJİ', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=39'),
                       ( u'3. BOYUT', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=172'),
                       ( u'HAYAT BİLGİSİ', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=283'),
                       ( u'İŞ DÜNYASI', u'http://www.aksiyon.com.tr/aksiyon/rss?sectionId=283'),
-
-
                         ]
 
-    def print_version(self, url):
+    #def print_version(self, url):
-     return url.replace('http://www.aksiyon.com.tr/aksiyon/newsDetail_getNewsById.action?load=detay&', 'http://www.aksiyon.com.tr/aksiyon/mobile_detailn.action?')
+     #return url.replace('http://www.aksiyon.com.tr/aksiyon/newsDetail_getNewsById.action?load=detay&', 'http://www.aksiyon.com.tr/aksiyon/mobile_detailn.action?')
 

recipes/ap.recipe

@@ -10,6 +10,8 @@ class AssociatedPress(BasicNewsRecipe):
     use_embedded_content   = False
     language = 'en'
     no_stylesheets = True
+    auto_cleanup = True
+#    auto_cleanup_keep = '//td[@class="ap-smallphoto-td-image"]'
     max_articles_per_feed = 15
 
 
@@ -20,13 +22,13 @@ class AssociatedPress(BasicNewsRecipe):
     ]
 
 
-    keep_only_tags = [ dict(name='div', attrs={'class':['body']}),
+    #keep_only_tags = [ dict(name='table', attrs={'class':['ap-story-table hnews hentry item']}),
-                       dict(name='div', attrs={'class':['entry-content']}),
+                       ##dict(name='div', attrs={'class':['entry-content']}),
-                       ]
+                       #]
-    remove_tags = [dict(name='table', attrs={'class':['ap-video-table','ap-htmlfragment-table','ap-htmltable-table']}),
+    #remove_tags = [dict(name='td', attrs={'class':['ap-mediabox-td']}),
-                   dict(name='span', attrs={'class':['apCaption','tabletitle']}),
+                   #dict(name='table', attrs={'class':['ap-htmltable-table', 'ap-htmltable-table', 'ap-mediabox-table']}),
-                   dict(name='td', attrs={'bgcolor':['#333333']}),
+                   ##dict(name='td', attrs={'bgcolor':['#333333']}),
-                  ]
+                  #]
     extra_css = '''
                .headline{font-family:Verdana,Arial,Helvetica,sans-serif;font-weight:bold;}
                .bline{color:#003366;}

recipes/baltimore_sun.recipe

@@ -11,10 +11,9 @@ class BaltimoreSun(BasicNewsRecipe):
 
     title       = 'The Baltimore Sun'
     __author__ = 'Josh Hall'
-
     description = 'Complete local news and blogs from Baltimore'
     language    = 'en'
-    version     = 2
+    version     = 2.1
     oldest_article = 1
     max_articles_per_feed = 100
     use_embedded_content = False
@@ -22,6 +21,7 @@ class BaltimoreSun(BasicNewsRecipe):
     remove_javascript    = True
     recursions           = 1
 
+    ignore_duplicate_articles = {'title'}
     keep_only_tags = [dict(name='div', attrs={'class':["story","entry-asset asset hentry"]}),
                       dict(name='div', attrs={'id':["pagebody","story","maincontentcontainer"]}),
                            ]
@@ -201,3 +201,5 @@ class BaltimoreSun(BasicNewsRecipe):
             tag.extract()
         for tag in soup.findAll('font', dict(attrs={'id':["cr-other-headlines"]})):
             tag.extract()
+
+        return soup

recipes/benchmark_pl.recipe

@@ -12,9 +12,9 @@ class BenchmarkPl(BasicNewsRecipe):
     max_articles_per_feed = 100
     no_stylesheets=True
     preprocess_regexps = [(re.compile(ur'<h3><span style="font-size: small;">&nbsp;Zobacz poprzednie <a href="http://www.benchmark.pl/news/zestawienie/grupa_id/135">Opinie dnia:</a></span>.*</body>', re.DOTALL|re.IGNORECASE), lambda match: '</body>'), (re.compile(ur'Więcej o .*?</ul>', re.DOTALL|re.IGNORECASE), lambda match: '')]
-    keep_only_tags=[dict(name='div', attrs={'class':['m_zwykly', 'gallery']})]
+    keep_only_tags=[dict(name='div', attrs={'class':['m_zwykly', 'gallery']}), dict(id='article')]
     remove_tags_after=dict(name='div', attrs={'class':'body'})
-    remove_tags=[dict(name='div', attrs={'class':['kategoria', 'socialize', 'thumb', 'panelOcenaObserwowane', 'categoryNextToSocializeGallery', 'breadcrumb']}), dict(name='table', attrs={'background':'http://www.benchmark.pl/uploads/backend_img/a/fotki_newsy/opinie_dnia/bg.png'}), dict(name='table', attrs={'width':'210', 'cellspacing':'1', 'cellpadding':'4', 'border':'0', 'align':'right'})]
+    remove_tags=[dict(name='div', attrs={'class':['kategoria', 'socialize', 'thumb', 'panelOcenaObserwowane', 'categoryNextToSocializeGallery', 'breadcrumb', 'footer', 'moreTopics']}), dict(name='table', attrs={'background':'http://www.benchmark.pl/uploads/backend_img/a/fotki_newsy/opinie_dnia/bg.png'}), dict(name='table', attrs={'width':'210', 'cellspacing':'1', 'cellpadding':'4', 'border':'0', 'align':'right'})]
     INDEX= 'http://www.benchmark.pl'
     feeds          = [(u'Aktualności', u'http://www.benchmark.pl/rss/aktualnosci-pliki.xml'), 
                           (u'Testy i recenzje', u'http://www.benchmark.pl/rss/testy-recenzje-minirecenzje.xml')]

recipes/biolog_pl.recipe

@@ -13,6 +13,7 @@ class Biolog_pl(BasicNewsRecipe):
     masthead_url= 'http://www.biolog.pl/naukowy,portal,biolog.png'
     cover_url='http://www.biolog.pl/naukowy,portal,biolog.png'
     no_stylesheets = True
+    ignore_duplicate_articles = {'title', 'url'}
     #keeps_only_tags=[dict(id='main')]
     remove_tags_before=dict(id='main')
     remove_tags_after=dict(name='a', attrs={'name':'komentarze'})

recipes/boston.com.recipe

@@ -15,7 +15,8 @@ class BusinessStandard(BasicNewsRecipe):
     no_stylesheets         = True
     delay                  = 1
     use_embedded_content   = False
-    encoding               = 'cp1252'
+    auto_cleanup = True
+    encoding               = 'utf-8'
     publisher              = 'Boston'
     category               = 'news, boston, usa, world'
     language               = 'en'
@@ -30,23 +31,23 @@ class BusinessStandard(BasicNewsRecipe):
                             ,'publisher'       : publisher
                          }
 
-    keep_only_tags   = [dict(attrs={'id':['INDblogEntry','blogEntry','articleHeader','articleGraphs','galleryShell']})]
+    #keep_only_tags   = [dict(attrs={'id':['INDblogEntry','blogEntry','articleHeader','articleGraphs','galleryShell']})]
-    remove_tags      = [
+    #remove_tags      = [
-                          dict(name=['object','link','script','iframe'])
+                          #dict(name=['object','link','script','iframe'])
-                         ,dict(attrs={'id':['blogheadTools','bdc_emailWidget','tools','relatedContent']})
+                         #,dict(attrs={'id':['blogheadTools','bdc_emailWidget','tools','relatedContent']})
-                       ]
+                       #]
 
     feeds = [
                (u'Top Stories'  , u'http://feeds.boston.com/boston/topstories'              )
-              ,(u'Patriots news', u'http://feeds.boston.com/boston/sports/football/patriots')
+              ,(u'Patriots news', u'http://feeds.boston.com/boston/sports/football/patriots/patriots_rss')
               ,(u'National news', u'http://feeds.boston.com/boston/news/nation'             )
               ,(u'World news'   , u'http://feeds.boston.com/boston/news/world'              )
             ]
 
-    def print_version(self, url):
+    #def print_version(self, url):
-        return url + '?page=full'
+        #return url + '?page=full'
 
-    def get_article_url(self, article):
+    #def get_article_url(self, article):
-        rawarticle = article.get('guid', None)
+        #rawarticle = article.get('guid', None)
-        return rawarticle.rpartition('?')[0]
+        #return rawarticle.rpartition('?')[0]
 

recipes/cgm_pl.recipe

@@ -13,11 +13,11 @@ class CGM(BasicNewsRecipe):
     use_embedded_content   = False
     remove_empty_feeds= True
     max_articles_per_feed = 100
-    no_stylesheers=True
+    no_stylesheets = True
     extra_css      = 'div {color:black;} strong {color:black;} span {color:black;} p {color:black;} h2 {color:black;}'
     remove_tags_before=dict(id='mainContent')
     remove_tags_after=dict(name='div', attrs={'class':'fbContainer'})
-    remove_tags=[dict(name='div', attrs={'class':'fbContainer'}),
+    remove_tags=[dict(name='div', attrs={'class':['fbContainer', 'socials']}),
           dict(name='p', attrs={'class':['tagCloud', 'galleryAuthor']}),
           dict(id=['movieShare', 'container'])]
     feeds          = [(u'Informacje', u'http://www.cgm.pl/rss.xml'), (u'Polecamy', u'http://www.cgm.pl/rss,4,news.xml'),

recipes/daily_mirror.recipe

@@ -7,7 +7,7 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
     description = 'News as provided by The Daily Mirror -UK'
 
     __author__ = 'Dave Asbury'
-    # last updated 8/6/12
+    # last updated 19/10/12
     language = 'en_GB'
     #cover_url = 'http://yookeo.com/screens/m/i/mirror.co.uk.jpg'
 
@@ -15,10 +15,12 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
 
 
     oldest_article = 1
-    max_articles_per_feed = 12
+    max_articles_per_feed = 1
     remove_empty_feeds = True
     remove_javascript     = True
     no_stylesheets = True
+    ignore_duplicate_articles = {'title'}
+
    # auto_cleanup = True
     #conversion_options = { 'linearize_tables' : True }
 
@@ -60,11 +62,12 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
 
            # example of commented out feed not needed ,(u'Travel','http://www.mirror.co.uk/advice/travel/rss.xml')
   ]
-    extra_css  = '''
+    extra_css = '''
-                           h1{ font-size:medium;}
+                    h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
-    body{ text-align: justify; font-family:Arial,Helvetica,sans-serif; font-size:11px; font-size-adjust:none; font-stretch:normal; font-style:normal; font-variant:normal; font-weight:normal;}
+                    h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
-                           img { display:block}
+                    p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
-                     '''#
+                    body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
+		'''
 
     def get_cover_url(self):
         soup = self.index_to_soup('http://www.politicshome.com/uk/latest_frontpage.html')
@@ -75,8 +78,10 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
 #cov2 now contains url of the page containing pic
         soup = self.index_to_soup(cov2)
         cov = soup.find(attrs={'id' : 'large'})
-        cov2 = str(cov)
+        cov=str(cov)
-        cov2=cov2[27:-18]
+        cov2 =  re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', cov)
+        cov2 = str(cov2)
+        cov2=cov2[2:len(cov2)-2]
         #cov2 now is pic url, now  go back to original function
         br = browser()
         br.set_handle_redirect(False)

recipes/dobreprogamy.recipe

@@ -19,7 +19,7 @@ class Dobreprogramy_pl(BasicNewsRecipe):
     max_articles_per_feed = 100
     preprocess_regexps = [(re.compile(ur'<div id="\S+360pmp4">Twoja przeglądarka nie obsługuje Flasha i HTML5 lub wyłączono obsługę JavaScript...</div>'), lambda match: '') ]
     keep_only_tags=[dict(attrs={'class':['news', 'entry single']})]
-    remove_tags = [dict(name='div', attrs={'class':['newsOptions', 'noPrint', 'komentarze', 'tags  font-heading-master']})]
+    remove_tags = [dict(attrs={'class':['newsOptions', 'noPrint', 'komentarze', 'tags  font-heading-master']}), dict(id='komentarze')]
     #remove_tags = [dict(name='div', attrs={'class':['komentarze', 'block', 'portalInfo', 'menuBar', 'topBar']})]
     feeds = [(u'Aktualności', 'http://feeds.feedburner.com/dobreprogramy/Aktualnosci'),
                  ('Blogi', 'http://feeds.feedburner.com/dobreprogramy/BlogCzytelnikow')]

recipes/dzieje_pl.recipe

@@ -12,9 +12,8 @@ class Dzieje(BasicNewsRecipe):
     max_articles_per_feed = 100
     remove_javascript=True
     no_stylesheets= True
-    remove_tags_before= dict(name='h1', attrs={'class':'title'})
+    keep_only_tags = [dict(name='h1', attrs={'class':'title'}), dict(id='content-area')]
-    remove_tags_after= dict(id='dogory')
+    remove_tags = [dict(attrs={'class':'field field-type-computed field-field-tagi'}), dict(id='dogory')]
-    remove_tags=[dict(id='dogory')]
     feeds          = [(u'Dzieje', u'http://dzieje.pl/rss.xml')]
 
 

recipes/dziennik_pl.recipe

@@ -15,6 +15,7 @@ class Dziennik_pl(BasicNewsRecipe):
     max_articles_per_feed = 100
     remove_javascript=True
     remove_empty_feeds=True
+    ignore_duplicate_articles = {'title', 'url'}
     extra_css= 'ul {list-style: none; padding: 0; margin: 0;} li {float: left;margin: 0 0.15em;}'
     preprocess_regexps     = [(re.compile("Komentarze:"), lambda m: ''), (re.compile('<p><strong><a href=".*?">&gt;&gt;&gt; CZYTAJ TAKŻE: ".*?"</a></strong></p>'), lambda m: '')]
     keep_only_tags=[dict(id='article')]
@@ -59,8 +60,6 @@ class Dziennik_pl(BasicNewsRecipe):
                 appendtag.find('div', attrs={'class':'article_paginator'}).extract()
 
 
-
-
     def preprocess_html(self, soup):
          self.append_page(soup, soup.body)
          return soup

recipes/film_web.recipe

@@ -13,12 +13,12 @@ class FilmWebPl(BasicNewsRecipe):
     max_articles_per_feed = 100
     no_stylesheets= True
     remove_empty_feeds=True
+    ignore_duplicate_articles = {'title', 'url'}
     preprocess_regexps = [(re.compile(u'\(kliknij\,\ aby powiększyć\)', re.IGNORECASE), lambda m: ''), ]#(re.compile(ur' | ', re.IGNORECASE), lambda m: '')]
     extra_css      = '.hdrBig {font-size:22px;} ul {list-style-type:none; padding: 0; margin: 0;}'
     remove_tags= [dict(name='div', attrs={'class':['recommendOthers']}), dict(name='ul', attrs={'class':'fontSizeSet'}), dict(attrs={'class':'userSurname anno'})]
     keep_only_tags= [dict(name='h1', attrs={'class':['hdrBig', 'hdrEntity']}), dict(name='div', attrs={'class':['newsInfo', 'newsInfoSmall', 'reviewContent description']})]
-    feeds          = [(u'Wszystkie newsy', u'http://www.filmweb.pl/feed/news/latest'),
+    feeds          = [(u'News / Filmy w produkcji', 'http://www.filmweb.pl/feed/news/category/filminproduction'),
-                         (u'News / Filmy w produkcji', 'http://www.filmweb.pl/feed/news/category/filminproduction'),
                          (u'News / Festiwale, nagrody i przeglądy', u'http://www.filmweb.pl/feed/news/category/festival'),
                          (u'News / Seriale', u'http://www.filmweb.pl/feed/news/category/serials'),
                          (u'News / Box office', u'http://www.filmweb.pl/feed/news/category/boxoffice'),
@@ -41,7 +41,6 @@ class FilmWebPl(BasicNewsRecipe):
         if skip_tag is not None:
             return self.index_to_soup(skip_tag['href'], raw=True)
 
-
     def preprocess_html(self, soup):
         for a in soup('a'):
             if a.has_key('href') and 'http://' not in a['href'] and 'https://' not in a['href']:

recipes/forsal.recipe

@@ -0,0 +1,23 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+import re
+class ForsalPL(BasicNewsRecipe):
+    title          = u'Forsal.pl'
+    __author__        = 'fenuks'
+    description   = u'Na portalu finansowym Forsal.pl znajdziesz najświeższe wiadomości finansowe i analizy. Kliknij i poznaj aktualne kursy walut, notowania giełdowe oraz inne wiadomości ze świata finansów.'
+    category       = 'economy, finance'
+    language       = 'pl'
+    oldest_article = 7
+    max_articles_per_feed = 100
+    use_embedded_content = False
+    ignore_duplicate_articles = {'title', 'url'}
+    cover_url = 'http://www.bizneswnieruchomosciach.pl/wp-content/uploads/2010/07/logo_forsal.jpg'
+    no_stylesheets = True
+    remove_tags = [dict(name='div', attrs={'class':'related'}), dict(name='img', attrs={'title':'Forsal'})]
+    feeds          = [(u'Najnowsze', u'http://forsal.pl/atom/najnowsze'), (u'Tylko na forsal.pl', u'http://forsal.pl/atom/tagi/forsal'), (u'Publicystyka', u'http://forsal.pl/atom/tagi/opinia'), (u'Bloomberg', u'http://forsal.pl/atom/tagi/bloomberg'), (u'Financial Times', u'http://forsal.pl/atom/tagi/financial_times'), (u'Gie\u0142da', u'http://forsal.pl/atom/tagi/gielda'), (u'Waluty', u'http://forsal.pl/atom/tagi/waluty'), (u'Surowce', u'http://forsal.pl/atom/tagi/surowce'), (u'Komenarze finasnowe', u'http://forsal.pl/atom/tagi/komentarz'), (u'Komentarze gie\u0142dowe', u'http://forsal.pl/atom/tagi/komentarz;gielda'), (u'Komentarze walutowe', u'http://forsal.pl/atom/tagi/komentarz;waluty'), (u'Makroekonomia', u'http://forsal.pl/atom/tagi/makroekonomia'), (u'Handel', u'http://forsal.pl/atom/tagi/handel'), (u'Nieruchomo\u015bci', u'http://forsal.pl/atom/tagi/nieruchomosci'), (u'Motoryzacja', u'http://forsal.pl/atom/tagi/motoryzacja'), (u'Finanse', u'http://forsal.pl/atom/tagi/finanse'), (u'Transport', u'http://forsal.pl/atom/tagi/transport'), (u'Media', u'http://forsal.pl/atom/tagi/media'), (u'Telekomunikacja', u'http://forsal.pl/atom/tagi/telekomunikacja'), (u'Energetyka', u'http://forsal.pl/atom/tagi/energetyka'), (u'Przemys\u0142', u'http://forsal.pl/atom/tagi/przemysl'), (u'Moja firma', u'http://forsal.pl/atom/tagi/moja_firma')]
+
+    def print_version(self, url):
+        url_id = re.search(ur'/[0-9]+,', url)
+        if url_id:
+            return 'http://forsal.pl/drukowanie' + url_id.group(0)[:-1]
+        else:
+            return url

recipes/gazeta_wyborcza.recipe

@@ -15,15 +15,28 @@ class Gazeta_Wyborcza(BasicNewsRecipe):
     max_articles_per_feed = 100
     remove_javascript=True
     no_stylesheets=True
-    remove_tags_before=dict(id='k0')
+    ignore_duplicate_articles = {'title', 'url'}
-    remove_tags_after=dict(id='banP4')
+    keep_only_tags = dict(id=['gazeta_article', 'article'])
-    remove_tags=[dict(name='div', attrs={'class':'rel_box'}), dict(attrs={'class':['date', 'zdjP', 'zdjM', 'pollCont', 'rel_video', 'brand', 'txt_upl']}), dict(name='div', attrs={'id':'footer'})]
+    remove_tags_after = dict(id='gazeta_article_share')
+    remove_tags = [dict(attrs={'class':['artReadMore', 'gazeta_article_related_new', 'txt_upl']}), dict(id=['gazeta_article_likes', 'gazeta_article_tools', 'rel', 'gazeta_article_tags', 'gazeta_article_share', 'gazeta_article_brand', 'gazeta_article_miniatures'])]
+
     feeds          = [(u'Kraj', u'http://rss.feedsportal.com/c/32739/f/530266/index.rss'), (u'\u015awiat', u'http://rss.feedsportal.com/c/32739/f/530270/index.rss'),
 	(u'Wyborcza.biz', u'http://wyborcza.biz/pub/rss/wyborcza_biz_wiadomosci.htm'),
 	(u'Komentarze', u'http://rss.feedsportal.com/c/32739/f/530312/index.rss'),
 	(u'Kultura', u'http://rss.gazeta.pl/pub/rss/gazetawyborcza_kultura.xml'),
-	(u'Nauka', u'http://rss.feedsportal.com/c/32739/f/530269/index.rss'), (u'Opinie', u'http://rss.gazeta.pl/pub/rss/opinie.xml'), (u'Gazeta \u015awi\u0105teczna', u'http://rss.feedsportal.com/c/32739/f/530431/index.rss'), (u'Du\u017cy Format', u'http://rss.feedsportal.com/c/32739/f/530265/index.rss'), (u'Witamy w Polsce', u'http://rss.feedsportal.com/c/32739/f/530476/index.rss'), (u'M\u0119ska Muzyka', u'http://rss.feedsportal.com/c/32739/f/530337/index.rss'), (u'Lata Lec\u0105', u'http://rss.feedsportal.com/c/32739/f/530326/index.rss'), (u'Solidarni z Tybetem', u'http://rss.feedsportal.com/c/32739/f/530461/index.rss'), (u'W pon. - \u017bakowski', u'http://rss.feedsportal.com/c/32739/f/530491/index.rss'), (u'We wt. - Kolenda-Zalewska', u'http://rss.feedsportal.com/c/32739/f/530310/index.rss'), (u'\u015aroda w \u015brod\u0119', u'http://rss.feedsportal.com/c/32739/f/530428/index.rss'), (u'W pi\u0105tek - Olejnik', u'http://rss.feedsportal.com/c/32739/f/530364/index.rss'), (u'Nekrologi', u'http://rss.feedsportal.com/c/32739/f/530358/index.rss')
+	(u'Nauka', u'http://rss.feedsportal.com/c/32739/f/530269/index.rss'), 
-                     ]
+	(u'Opinie', u'http://rss.gazeta.pl/pub/rss/opinie.xml'), 
+	(u'Gazeta \u015awi\u0105teczna', u'http://rss.feedsportal.com/c/32739/f/530431/index.rss'), 
+	#(u'Du\u017cy Format', u'http://rss.feedsportal.com/c/32739/f/530265/index.rss'), 
+	(u'Witamy w Polsce', u'http://rss.feedsportal.com/c/32739/f/530476/index.rss'), 
+	(u'M\u0119ska Muzyka', u'http://rss.feedsportal.com/c/32739/f/530337/index.rss'), 
+	(u'Lata Lec\u0105', u'http://rss.feedsportal.com/c/32739/f/530326/index.rss'), 
+	(u'Solidarni z Tybetem', u'http://rss.feedsportal.com/c/32739/f/530461/index.rss'), 
+	(u'W pon. - \u017bakowski', u'http://rss.feedsportal.com/c/32739/f/530491/index.rss'), 
+	(u'We wt. - Kolenda-Zalewska', u'http://rss.feedsportal.com/c/32739/f/530310/index.rss'), 
+	(u'\u015aroda w \u015brod\u0119', u'http://rss.feedsportal.com/c/32739/f/530428/index.rss'), 
+	(u'W pi\u0105tek - Olejnik', u'http://rss.feedsportal.com/c/32739/f/530364/index.rss')
+          ]
 
     def skip_ad_pages(self, soup):
           tag=soup.find(name='a', attrs={'class':'btn'})

recipes/gildia_pl.recipe

@@ -1,4 +1,5 @@
 from calibre.web.feeds.news import BasicNewsRecipe
+import re
 
 class Gildia(BasicNewsRecipe):
     title          = u'Gildia.pl'
@@ -11,6 +12,8 @@ class Gildia(BasicNewsRecipe):
     max_articles_per_feed = 100
     remove_empty_feeds=True
     no_stylesheets=True
+    ignore_duplicate_articles = {'title', 'url'}
+    preprocess_regexps = [(re.compile(ur'</?sup>'), lambda match: '') ]
     remove_tags=[dict(name='div', attrs={'class':'backlink'}), dict(name='div', attrs={'class':'im_img'}), dict(name='div', attrs={'class':'addthis_toolbox addthis_default_style'})]
     keep_only_tags=dict(name='div', attrs={'class':'widetext'})
     feeds          = [(u'Gry', u'http://www.gry.gildia.pl/rss'), (u'Literatura', u'http://www.literatura.gildia.pl/rss'), (u'Film', u'http://www.film.gildia.pl/rss'), (u'Horror', u'http://www.horror.gildia.pl/rss'), (u'Konwenty', u'http://www.konwenty.gildia.pl/rss'), (u'Plansz\xf3wki', u'http://www.planszowki.gildia.pl/rss'), (u'Manga i anime', u'http://www.manga.gildia.pl/rss'), (u'Star Wars', u'http://www.starwars.gildia.pl/rss'), (u'Techno', u'http://www.techno.gildia.pl/rss'), (u'Historia', u'http://www.historia.gildia.pl/rss'), (u'Magia', u'http://www.magia.gildia.pl/rss'), (u'Bitewniaki', u'http://www.bitewniaki.gildia.pl/rss'), (u'RPG', u'http://www.rpg.gildia.pl/rss'), (u'LARP', u'http://www.larp.gildia.pl/rss'), (u'Muzyka', u'http://www.muzyka.gildia.pl/rss'), (u'Nauka', u'http://www.nauka.gildia.pl/rss')]
@@ -18,10 +21,9 @@ class Gildia(BasicNewsRecipe):
 
     def skip_ad_pages(self, soup):
         content = soup.find('div', attrs={'class':'news'})
-        skip_tag= content.findAll(name='a')
+        if 'recenzj' in soup.title.string.lower():
-        if skip_tag is not None:
+            for link in content.findAll(name='a'):
-            for link in skip_tag:
+                if 'recenzj' in link['href']:
-                if 'recenzja' in link['href']:
                     self.log.warn('odnosnik')
                     self.log.warn(link['href'])
                     return self.index_to_soup(link['href'], raw=True)

recipes/gram_pl.recipe

@@ -12,7 +12,7 @@ class Gram_pl(BasicNewsRecipe):
     no_stylesheets= True
     extra_css = 'h2 {font-style: italic;  font-size:20px;} .picbox div {float: left;}'
     cover_url=u'http://www.gram.pl/www/01/img/grampl_zima.png'
-    remove_tags= [dict(name='p', attrs={'class':['extraText', 'must-log-in']}), dict(attrs={'class':['el', 'headline', 'post-info', 'entry-footer clearfix']}), dict(name='div', attrs={'class':['twojaOcena', 'comment-body', 'comment-author vcard', 'comment-meta commentmetadata', 'tw_button', 'entry-comment-counter', 'snap_nopreview sharing robots-nocontent']}), dict(id=['igit_rpwt_css', 'comments', 'reply-title', 'igit_title'])]
+    remove_tags= [dict(name='p', attrs={'class':['extraText', 'must-log-in']}), dict(attrs={'class':['el', 'headline', 'post-info', 'entry-footer clearfix']}), dict(name='div', attrs={'class':['twojaOcena', 'comment-body', 'comment-author vcard', 'comment-meta commentmetadata', 'tw_button', 'entry-comment-counter', 'snap_nopreview sharing robots-nocontent', 'sharedaddy sd-sharing-enabled']}), dict(id=['igit_rpwt_css', 'comments', 'reply-title', 'igit_title'])]
     keep_only_tags= [dict(name='div', attrs={'class':['main', 'arkh-postmetadataheader', 'arkh-postcontent', 'post', 'content', 'news_header', 'news_subheader', 'news_text']}), dict(attrs={'class':['contentheading', 'contentpaneopen']}), dict(name='article')]
     feeds          = [(u'Informacje', u'http://www.gram.pl/feed_news.asp'),
 	      (u'Publikacje', u'http://www.gram.pl/feed_news.asp?type=articles'),

recipes/gry_online_pl.recipe

@@ -12,8 +12,8 @@ class GryOnlinePl(BasicNewsRecipe):
     cover_url='http://www.gry-online.pl/im/gry-online-logo.png'
     max_articles_per_feed = 100
     no_stylesheets= True
-    keep_only_tags=[dict(name='div', attrs={'class':'gc660'})]
+    keep_only_tags=[dict(name='div', attrs={'class':['gc660', 'gc660 S013']})]
-    remove_tags=[dict({'class':['nav-social', 'add-info', 'smlb', 'lista lista3 lista-gry', 'S013po', 'zm_gfx_cnt_bottom', 'ocen-txt', 'wiecej-txt', 'wiecej-txt2']})]
+    remove_tags=[dict({'class':['nav-social', 'add-info', 'smlb', 'lista lista3 lista-gry', 'S013po', 'S013-npb', 'zm_gfx_cnt_bottom', 'ocen-txt', 'wiecej-txt', 'wiecej-txt2']})]
     feeds          = [(u'Newsy', 'http://www.gry-online.pl/rss/news.xml'), ('Teksty', u'http://www.gry-online.pl/rss/teksty.xml')]
 
 

recipes/konflikty_zbrojne.recipe

@@ -15,7 +15,6 @@ class Konflikty(BasicNewsRecipe):
     keep_only_tags=[dict(attrs={'class':['title1', 'image']}), dict(id='body')]
 
     feeds          = [(u'Aktualności', u'http://www.konflikty.pl/rss_aktualnosci_10.xml'),
-		(u'Artyku\u0142y', u'http://www.konflikty.pl/rss_artykuly_10.xml'),
 		(u'Historia', u'http://www.konflikty.pl/rss_historia_10.xml'),
 		(u'Militaria', u'http://www.konflikty.pl/rss_militaria_10.xml'),
 		(u'Relacje', u'http://www.konflikty.pl/rss_relacje_10.xml'),

recipes/lomza.recipe

@@ -7,7 +7,7 @@ class Lomza(BasicNewsRecipe):
     cover_url      = 'http://www.4lomza.pl/i/logo4lomza_m.jpg'
     language       = 'pl'
     oldest_article = 15
-    no_styleseets=True
+    no_stylesheets = True
     max_articles_per_feed = 100
     remove_tags=[dict(name='div', attrs={'class':['bxbanner', 'drukuj', 'wyslijznajomemu']})]
     keep_only_tags=[dict(name='div', attrs={'class':'wiadomosc'})]

recipes/lwn_weekly.recipe

@@ -8,15 +8,18 @@ lwn.net
 
 from calibre.web.feeds.news import BasicNewsRecipe
 import re
+import sys
 
 class WeeklyLWN(BasicNewsRecipe):
     title = 'LWN.net Weekly Edition'
     description = 'Weekly summary of what has happened in the free software world.'
     __author__ = 'Davide Cavalca'
     language = 'en'
-    site_url = 'http://lwn.net'
+    site_url = u'http://lwn.net'
 
     extra_css = 'pre,code,samp,kbd,tt { font-size: 80% }\nblockquote {margin-left:0 }\n* { color: black }\n'
+    no_stylesheets = True
+    remove_javascript = True
 
     cover_url = site_url + '/images/lcorner.png'
     #masthead_url = 'http://lwn.net/images/lcorner.png'
@@ -28,10 +31,13 @@ class WeeklyLWN(BasicNewsRecipe):
 
     preprocess_regexps = [
         # Remove the <hr> and "Log in to post comments"
-        (re.compile(r'<hr.*?comments[)]', re.DOTALL), lambda m: ''),
+        (re.compile(r'<hr [^>]+>\s*\n\s*.*?comments[)]'), lambda m: ''),
     ]
 
-    conversion_options = { 'linearize_tables' : True }
+    conversion_options = {
+        'linearize_tables' : True,
+        'no_inline_navbars': True,
+    }
 
     oldest_article = 7.0
     needs_subscription = 'optional'
@@ -60,8 +66,6 @@ class WeeklyLWN(BasicNewsRecipe):
         if url[-len(print_param):] != print_param:
             url += print_param
 
-        #import sys
-        #print >>sys.stderr, "*** print_version(url):", url
         return url
 
     def parse_index(self):
@@ -70,61 +74,69 @@ class WeeklyLWN(BasicNewsRecipe):
         else:
             index_url = self.print_version('/free/bigpage')
         soup = self.index_to_soup(index_url)
-        body = soup.body
+        curr = soup.body
 
         articles = {}
         ans = []
-        url_re = re.compile('^/Articles/')
+
+        section = soup.title.string
+        subsection = None
 
         while True:
-            tag_title = body.findNext(attrs={'class':'SummaryHL'})
+            curr = curr.findNext(attrs = {'class': ['SummaryHL', 'Cat1HL', 'Cat2HL'] })
-            if tag_title == None:
+
+            if curr == None:
                 break
 
-            tag_section = tag_title.findPrevious(attrs={'class':'Cat1HL'})
+            text = curr.contents[0].string
-            if tag_section == None:
-                section = 'Front Page'
-            else:
-                section = tag_section.string
 
-            tag_section2 = tag_title.findPrevious(attrs={'class':'Cat2HL'})
+            if 'Cat2HL' in curr.attrMap['class']:
-            if tag_section2 != None:
+                subsection = text
-                if tag_section2.findPrevious(attrs={'class':'Cat1HL'}) == tag_section:
-                    section = "%s: %s" %(section, tag_section2.string)
 
-            if section not in articles.keys():
+            elif 'Cat1HL' in curr.attrMap['class']:
-                articles[section] = []
+                section = text
-            if section not in ans:
+                subsection = None
-                ans.append(section)
 
-            body = tag_title
+            elif 'SummaryHL' in curr.attrMap['class']:
-            while True:
+                article_title = text
-                tag_url = body.findNext(name='a', attrs={'href':url_re})
+
-                if tag_url == None:
+                if subsection:
-                    break
+                    section_title = "%s: %s" % (section, subsection)
-                body = tag_url
-                if tag_url.string == None:
-                    continue
-                elif tag_url.string == 'Full Story':
-                    break
-                elif tag_url.string.startswith('Comments ('):
-                    break
                 else:
+                    section_title = section
+
+                # Most articles have anchors in their titles, *except* the security vulnerabilities
+                article_anchor = curr.findNext(name = 'a', attrs = { 'href': re.compile('^/Articles/') } )
+
+                if article_anchor:
+                    article_url = article_anchor.get('href')
+                    if not article_url:
+                        print >>sys.stderr, 'article_url is None for article_anchor "%s": "%s"' \
+                                % (str(article_anchor), article_title)
+                        continue
+
+                else:
+                    print >>sys.stderr, 'article_anchor is None for "%s"; skipping' % article_title
+                    article_url = None
                     continue
 
-            if tag_url == None:
+                if section_title not in articles:
-                break
+                    articles[section_title] = []
+                if section_title not in ans:
+                    ans.append(section_title)
 
+                articles[section_title].append({
+                        'url': article_url,
+                        'title': article_title,
+                        'description': '', 'content': '', 'date': '',
+                    })
 
-            article = dict(
+            else:
-                title=self.tag_to_string(tag_title),
+                print >>sys.stderr, "lwn_weekly.recipe: something bad happened; should not be able to reach this"
-                url=tag_url['href'],
-                description='', content='', date='')
-            articles[section].append(article)
 
-        ans = [(key, articles[key]) for key in ans if articles.has_key(key)]
+        ans = [(section, articles[section]) for section in ans if section in articles]
-        if not ans:
+        #from pprint import pprint
-            raise Exception('Could not find any articles.')
+        #pprint(ans)
 
         return ans
 

recipes/nowy_ekran.recipe

@@ -0,0 +1,16 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+class NowyEkran(BasicNewsRecipe):
+    title          = u'Nowy ekran'
+    oldest_article = 7
+    max_articles_per_feed = 100
+    no_stylesheets= True
+    __author__        = 'fenuks'
+    description   = u'Niezależny serwis społeczności blogerów'
+    category       = 'blog'
+    language       = 'pl'
+    masthead_url='http://s.nowyekran.pl/gfx/ekran-big.gif'
+    cover_url= 'http://s.nowyekran.pl/gfx/ekran-big.gif'
+    remove_tags_before = dict(name='div', attrs={'class':'post_detal'})
+    remove_tags_after = dict(name='div', attrs={'class':'post_footer'})
+    remove_tags=[dict(name='span', attrs={'class':'ico ico_comments'}), dict(name='div', attrs={'class':'post_footer'}), dict(name='a', attrs={'class':'getpdf'})]
+    feeds          = [(u'Najnowsze notki', u'http://www.nowyekran.pl/RSS/')]

recipes/nypost.recipe

@@ -17,6 +17,7 @@ class NYPost(BasicNewsRecipe):
     no_stylesheets        = True
     encoding              = 'utf8'
     use_embedded_content  = False
+    auto_cleanup = True
     language              = 'en'
     masthead_url          = 'http://www.nypost.com/rw/SysConfig/WebPortal/nypost/images/nyp_logo_230x32.gif'
     extra_css             = ' body{font-family: Arial,Helvetica,sans-serif } img{margin-bottom: 0.4em} '
@@ -28,7 +29,7 @@ class NYPost(BasicNewsRecipe):
                         , 'language'  : language
                         }
 
-    keep_only_tags=[dict(name='div', attrs={'id':'story'})]
+    #keep_only_tags=[dict(name='div', attrs={'id':'story'})]
 
     feeds = [(u'Articles', u'http://www.nypost.com/rss/all_section.xml')]
 

recipes/pc_world.recipe

@@ -35,6 +35,7 @@ class pcWorld(BasicNewsRecipe):
 
     remove_javascript     = True
     no_stylesheets        = True
+    auto_cleanup = True
 
     def get_obfuscated_article(self, url):
         br = self.get_browser()
@@ -48,16 +49,16 @@ class pcWorld(BasicNewsRecipe):
         self.temp_files[-1].close()
         return self.temp_files[-1].name
 
-    keep_only_tags     = [
+    #keep_only_tags     = [
-                            dict(name='div', attrs={'class':'article'})
+                            #dict(name='div', attrs={'class':'article'})
-                        ]
+                        #]
-    remove_tags        = [
+    #remove_tags        = [
-                            dict(name='div', attrs={'class':['toolBar','mac_tags','toolBar  btmTools','recommend longRecommend','recommend shortRecommend','textAds']}),
+                            #dict(name='div', attrs={'class':['toolBar','mac_tags','toolBar  btmTools','recommend longRecommend','recommend shortRecommend','textAds']}),
-                            dict(name='div', attrs={'id':['sidebar','comments','mac_tags']}),
+                            #dict(name='div', attrs={'id':['sidebar','comments','mac_tags']}),
-                            dict(name='ul', attrs={'class':['tools', 'tools clearfix']}),
+                            #dict(name='ul', attrs={'class':['tools', 'tools clearfix']}),
-                            dict(name='li', attrs={'class':'sub'}),
+                            #dict(name='li', attrs={'class':'sub'}),
-                            dict(name='p', attrs={'id':'userDesire'})
+                            #dict(name='p', attrs={'id':'userDesire'})
-                        ]
+                        #]
     feeds          = [
                        (u'PCWorld Headlines', u'http://feeds.pcworld.com/pcworld/latestnews'),
                        (u'How-To', u'http://feeds.pcworld.com/pcworld/update/howto'),

recipes/polska_times.recipe

@@ -1,5 +1,5 @@
 from calibre.web.feeds.news import BasicNewsRecipe
-class Polska_times(BasicNewsRecipe):
+class PolskaTimes(BasicNewsRecipe):
     title          = u'Polska Times'
     __author__        = 'fenuks'
     description   = u'Internetowe wydanie dziennika ogólnopolskiego Polska The Times. Najświeższe informacje: wydarzenia w kraju i na świecie, reportaże, poradniki, opinie.'
@@ -10,6 +10,7 @@ class Polska_times(BasicNewsRecipe):
     max_articles_per_feed = 100
     remove_emty_feeds= True
     no_stylesheets = True
+    ignore_duplicate_articles = {'title', 'url'}
     #preprocess_regexps = [(re.compile(ur'<b>Czytaj także:.*?</b>', re.DOTALL), lambda match: ''), (re.compile(ur',<b>Czytaj też:.*?</b>', re.DOTALL), lambda match: ''), (re.compile(ur'<b>Zobacz także:.*?</b>', re.DOTALL), lambda match: ''), (re.compile(ur'<center><h4><a.*?</a></h4></center>', re.DOTALL), lambda match: ''), (re.compile(ur'<b>CZYTAJ TEŻ:.*?</b>', re.DOTALL), lambda match: ''), (re.compile(ur'<b>CZYTAJ WIĘCEJ:.*?</b>', re.DOTALL), lambda match: ''), (re.compile(ur'<b>CZYTAJ TAKŻE:.*?</b>', re.DOTALL), lambda match: ''), (re.compile(ur'<b>\* CZYTAJ KONIECZNIE:.*', re.DOTALL), lambda match: '</body>'), (re.compile(ur'<b>Nasze serwisy:</b>.*', re.DOTALL), lambda match: '</body>') ]
     remove_tags_after= dict(attrs={'src':'http://nm.dz.com.pl/dz.png'})
     remove_tags=[dict(id='mat-podobne'), dict(name='a', attrs={'class':'czytajDalej'}), dict(attrs={'src':'http://nm.dz.com.pl/dz.png'})]
@@ -23,6 +24,7 @@ class Polska_times(BasicNewsRecipe):
             nexturl=soup.find('a')['href']
             return self.index_to_soup(nexturl, raw=True)
 
+
     def get_cover_url(self):
         soup = self.index_to_soup('http://www.prasa24.pl/gazeta/metropolia-warszawska/')
         self.cover_url=soup.find(id='pojemnik').img['src']

recipes/puls_biznesu.recipe

@@ -0,0 +1,31 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+import re
+class PB_PL(BasicNewsRecipe):
+    title          = u'Puls Biznesu'
+    __author__        = 'fenuks'
+    language       = 'pl'
+    description = u'Puls Biznesu - biznes, ekonomia, giełda, inwestycje'
+    category = u'newspaper'
+    publication_type = u'newspaper'
+    encoding = 'utf-8'
+    #masthead_url = 'http://www.pb.pl/img/pb.png'
+    oldest_article = 7
+    max_articles_per_feed = 100
+    no_stylesheets = True
+    remove_empty_feeds = True
+    ignore_duplicate_articles = {'title', 'url'}
+    remove_tags_after = dict(name='div', attrs={'class':'news_content'})
+    feeds          = [(u'Wszystkie', u'http://www.pb.pl/atom'), (u'Puls inwestora', u'http://pulsinwestora.pb.pl/atom'), (u'Puls Firmy', u'http://firma.pb.pl/atom'), (u'PB Weekend', u'http://weekend.pb.pl/atom'), (u'Forum MPS', u'http://forummsp.pb.pl/atom'), (u'Moto', u'http://moto.pb.pl/atom'), (u'Kariera i praca', u'http://kariera.pb.pl/atom'),(u'Nieruchomości', u'http://nieruchomosci.pb.pl/atom'), (u'Samorządy', u'http://samorzady.pb.pl/atom'), (u'Tech', u'http://tech.pb.pl/atom'), (u'Energetyka', u'http://energetyka.pb.pl/atom'), (u'Retailing', u'http://retailing.pb.pl/atom'), (u'Puls medycyny', u'http://pulsmedycyny.pl/atom'), (u'Logistyka', u'http://logistyka.pb.pl/atom')]
+
+    def print_version(self, url):
+        article_id = re.search(r'(?P<id>\d+,\d+)', url)
+        if article_id:
+            return 'http://www.pb.pl/actionprint/' + article_id.group('id')
+        else:
+            return url
+
+    def get_cover_url(self):
+        soup = self.index_to_soup('http://archiwum.pb.pl/')
+        cover = soup.find(name='img', attrs={'class':'cover_picture'})
+        self.cover_url= cover['src']
+        return getattr(self, 'cover_url', self.cover_url)

recipes/richmond_times_dispatch.recipe

@@ -9,13 +9,14 @@ class RichmondTimesDispatch(BasicNewsRecipe):
                             and is also a default paper for rural regions of the state.  \
                             The RTD has published in some form for more than 150 years."
     __author__          = '_reader'
-    __date__            = '05 July 2012'
+    __date__            = '17 October 2012'
-    __version__         = '1.4'
+    __version__         = '1.6'
     cover_url           = 'http://static2.dukecms.com/va_tn/timesdispatch_com/site-media/img/icons/logo252x97.png'
     masthead_url            = 'http://static2.dukecms.com/va_tn/timesdispatch_com/site-media/img/icons/logo252x97.png'
     language            = 'en'
     oldest_article      = 1.5 #days
     max_articles_per_feed   = 100
+    ignore_duplicate_articles = { 'title', 'url' }
     needs_subscription  = False
     publisher           = 'timesdispatch.com'
     category            = 'news, commentary'
@@ -70,6 +71,7 @@ class RichmondTimesDispatch(BasicNewsRecipe):
     ('Local Business',  'http://www2.timesdispatch.com/list/feed/rss/local-business'),
     ('Politics',        'http://www2.timesdispatch.com/list/feed/rss/politics'),
     ('Virginia Politics',   'http://www2.timesdispatch.com/list/feed/rss/virginia-politics'),
+    ('History',   'http://www2.timesdispatch.com/feed/rss/special_section/news/history'),
     ('Sports',      'http://www2.timesdispatch.com/list/feed/rss/sports2'),
     ('Health',      'http://www2.timesdispatch.com/feed/rss/lifestyles/health_med_fit/'),
     ('Entertainment/Life',  'http://www2.timesdispatch.com/list/feed/rss/entertainment'),
@@ -78,13 +80,15 @@ class RichmondTimesDispatch(BasicNewsRecipe):
     ('Music',       'http://www2.timesdispatch.com/list/feed/rss/music'),
     ('Dining & Food',   'http://www2.timesdispatch.com/list/feed/rss/dining'),
     ('Home & Garden',   'http://www2.timesdispatch.com/list/feed/rss/home-and-garden/'),
-    #inactive('Travel',     'http://www2.timesdispatch.com/feed/rss/travel/'),
+    ('Travel',     'http://www2.timesdispatch.com/feed/rss/travel/'),
     ('Opinion',     'http://www2.timesdispatch.com/feed/rss/news/opinion/'),
     ('Editorials',      'http://www2.timesdispatch.com/list/feed/rss/editorial-desk'),
     ('Columnists and Blogs',    'http://www2.timesdispatch.com/list/feed/rss/news-columnists-blogs'),
     ('Opinion Columnists',  'http://www2.timesdispatch.com/list/feed/rss/opinion-editorial-columnists'),
     ('Letters to the Editor',   'http://www2.timesdispatch.com/list/feed/rss/opinion-letters'),
     ('Traffic',     'http://www2.timesdispatch.com/list/feed/rss/traffic'),
+    ('Drives',     'http://www2.timesdispatch.com/feed/rss/classifieds/transportation/'),
+
         ]
 
     def print_version(self,url):

recipes/spektrum.recipe

@@ -1,28 +1,57 @@
+__license__   = 'GPL v3'
+__copyright__ = ''
+'''
+Fetch RSS-Feeds spektrum.de
+'''
 from calibre.web.feeds.recipes import BasicNewsRecipe
 class AdvancedUserRecipe1303841067(BasicNewsRecipe):
-    title          = u'Spektrum (der Wissenschaft)'
+    title                 = u'Spektrum der Wissenschaft'
-    __author__  = 'schuster'
+    __author__            = 'Armin Geller, Bratzzo, Rainer Zenz' # Update Bratzzo & AGE 2012-10-12
-    oldest_article = 7
+    description           = u'German  online portal of Spektrum der Wissenschaft'
+    publisher             = 'Spektrum der Wissenschaft Verlagsgesellschaft mbH'
+    category              = 'science news, Germany'
+
+    oldest_article        = 7
     max_articles_per_feed = 100
-    language               = 'de'
+    no_stylesheets        = True
-    cover_url = 'http://upload.wikimedia.org/wikipedia/de/3/3b/Spektrum_der_Wissenschaft_Logo.svg'
+    remove_javascript     = True
+    remove_empty_feeds    = True
+    language              = 'de_DE'
 
-    remove_tags = [dict(attrs={'class':['hauptnaviPkt gainlayout', 'hauptnaviButton', 'suchButton', 'suchbegriffKasten', 'loginButton', 'subnavigation', 'artikelInfoLeiste gainlayout', 'artikelTools', 'nurLetzteSeite', 'link', 'boxUnterArtikel', 'leserbriefeBlock', 'boxTitel', 'boxInhalt', 'sehrklein', 'boxabstand', 'werbeboxinhalt', 'rbabstand', 'bildlinks', 'rechtebox', 'denkmalbox', 'denkmalfrage']}),
+    #conversion_options    = {'base_font_size': 20}
-	dict(id=['pflip', 'verlagsleiste', 'bereich', 'bannerVertikal', 'headerLogoLink', 'kopf', 'topNavi', 'headerSchnellsuche', 'headerSchnellsucheWarten', 'navigation', 'navigationL', 'navigationR', 'inhalt', 'rechtespalte', 'sdwboxenshop', 'shopboxen', 'fuss']),
-	dict(name=['naservice'])]
 
-    def print_version(self,url):
+    # cover_url = 'http://upload.wikimedia.org/wikipedia/de/3/3b/Spektrum_der_Wissenschaft_Logo.svg' # old logo
-        newurl = url.replace('artikel/', 'sixcms/detail.php?id=')
+    cover_url             = 'http://upload.wikimedia.org/wikipedia/de/5/59/Spektrum-cover.jpg' # from Rainer Zenz
-        return newurl + '&_druckversion=1'
 
+    masthead_url          = 'http://www.spektrum.de/fm/861/spektrum.de.png'
 
+    extra_css      =  '''
+                      h1 {font-size: 1.6em; text-align: left}
+                      h2 {font-size: 1em; font-style: italic; font-weight: normal}
+                      h3 {font-size: 1.3em;text-align: left}
+                      h4, h5, h6, .heading, .hgroup {font-size: 1em;text-align: left}
+                      '''
 
-    feeds          = [(u'Spektrum der Wissenschaft', u'http://www.spektrum.de/artikel/982623'),
+    feeds = [
-                          (u'SpektrumDirekt', u'http://www.spektrumdirekt.de/artikel/996406'),
+              (u'Spektrum.de', u'http://www.spektrum.de/alias/rss/spektrum-de-rss-feed/996406'),
-                          (u'Sterne und Weltraum', u'http://www.astronomie-heute.de/artikel/865248'),
+              (u'Spektrum der Wissenschaft', u'http://www.spektrum.de/alias/rss/spektrum-der-wissenschaft-rss-feed/982623'),
-                          (u'Gehirn & Geist', u'http://www.gehirn-und-geist.de/artikel/982626'),
+              (u'Gehirn & Geist', u'http://www.spektrum.de/alias/rss/gehirn-geist-rss-feed/982626'),
-                          (u'epoc', u'http://www.epoc.de/artikel/982625')
+              (u'Epoc', u'http://www.spektrum.de/alias/rss/epoc-rss-feed/982625'),
+              (u'Sterne und Weltraum', u'http://www.spektrum.de/alias/rss/sterne-und-weltraum-rss-feed/865248'),
+              (u'Editional', u'http://www.spektrum.de/alias/rss/spektrum-de-editorial/996044'),
+              (u'Pressemitteilungen', u'http://www.spektrum.de/alias/rss/pressemitteilungen/995265'),
+             ]
 
-]
+    keep_only_tags = [
+                        dict(name='div', attrs={'class':'border-h clearfix article-top'}),
+                        dict(name='div', attrs={'class':'clearfix'}),
+                        dict(name='div', attrs={'class':'bilderrahmenlinks'}),
+                        dict(name='div', attrs={'class':'relcontainer'}),
+                      ]
 
-    filter_regexps = [r'ads\.doubleclick\.net']
+    remove_tags_after=dict(name='div', attrs={'class':['sidebar-box-head']})
+
+    remove_tags = [
+                    dict(attrs={'id':['recommend-article', 'dossierbox', 'cover', 'toc']}),
+                    dict(attrs={'class':['sidebar-box-full clearfix', 'linktotop' ]}),
+                  ]

recipes/spiders_web_pl.recipe

@@ -11,5 +11,5 @@ class SpidersWeb(BasicNewsRecipe):
     no_stylesheers=True
     max_articles_per_feed = 100
     keep_only_tags=[dict(id='Post')]
-    remove_tags=[dict(name='div', attrs={'class':['Comments', 'Shows', 'Post-Tags']})]
+    remove_tags=[dict(name='div', attrs={'class':['Comments', 'Shows', 'Post-Tags']}), dict(id='Author-Column')]
     feeds          = [(u'Wpisy', u'http://www.spidersweb.pl/feed')]

recipes/stopklatka.recipe

@@ -0,0 +1,25 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+import re
+class Stopklatka(BasicNewsRecipe):
+    title          = u'Stopklatka'
+    __author__        = 'fenuks'
+    description   = u'Stopklatka.pl to najdłużej działający polski portal filmowy. Baza filmów, seriali i aktorów, repertuar kin, program tv, wydarzenia ze świata filmu'
+    category       = 'movies'
+    language       = 'pl'
+    oldest_article = 7
+    masthead_url= 'http://img.stopklatka.pl/logo/logo-3.gif'
+    cover_url= 'http://img.stopklatka.pl/logo/logo-3.gif'
+    max_articles_per_feed = 100
+    no_stylesheets = True
+    preprocess_regexps = [(re.compile(ur'Wersja internetowa dostępna jest pod adresem:.*</body>', re.DOTALL), lambda match: '</body>'), (re.compile(ur'</?font.*?>', re.DOTALL), lambda match: '') ]
+    remove_empty_feeds = True
+    remove_tags = [dict(name='img', attrs={'alt':'logo'})]
+    feeds          = [(u'Wydarzenia', u'http://rss.stopklatka.pl/wydarzenia.rss')]
+
+    def print_version(self, url):
+        link_id = re.search(r'wi=(?P<id>\d+)', url)
+        if link_id:
+            return 'http://www.stopklatka.pl/narzedzia/drukuj.asp?typ=wydarzenie&id=' + link_id.group('id')
+        else:
+            return url
+

recipes/swiat_obrazu.recipe

@@ -19,7 +19,7 @@ class Swiat_Obrazu(BasicNewsRecipe):
         return url + ',drukuj'
 
     def image_url_processor(self, baseurl, url):
-        if 'http://' not in url or 'https://' not in url:
+        if 'http://' not in url and 'https://' not in url:
             return 'http://www.swiatobrazu.pl' + url[5:]
         else:
             return url

recipes/the_sun.recipe

@@ -1,4 +1,4 @@
-import random
+import re, random
 
 from calibre import browser
 from calibre.web.feeds.recipes import BasicNewsRecipe
@@ -8,7 +8,7 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     title          = u'The Sun UK'
     description = 'Articles from The Sun tabloid UK'
     __author__ = 'Dave Asbury'
-    # last updated 12/10/12 added starsons remove article code
+    # last updated 19/10/12 better cover fetch
     language = 'en_GB'
     oldest_article = 1
     max_articles_per_feed = 15
@@ -19,7 +19,7 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     remove_javascript     = True
     no_stylesheets = True
 
-    ignore_duplicate_articles = {'title'}
+    ignore_duplicate_articles = {'title','url'}
 
 
     extra_css  = '''
@@ -72,9 +72,10 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
         #cov2 now contains url of the page containing pic
         soup = self.index_to_soup(cov2)
         cov = soup.find(attrs={'id' : 'large'})
-        cov2 = str(cov)
+        cov=str(cov)
-        cov2=cov2[27:-18]
+        cov2 =  re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', cov)
-        #cov2 now is pic url, now  go back to original function
+        cov2 = str(cov2)
+        cov2=cov2[2:len(cov2)-2]
         br = browser()
         br.set_handle_redirect(False)
         try:

recipes/time_magazine.recipe

@@ -23,16 +23,15 @@ class Time(BasicNewsRecipe):
 
     keep_only_tags = [
             {
-                'class':['tout1', 'entry-content', 'external-gallery-img', 'image-meta']
+                'class':['primary-col', 'tout1']
             },
         ]
     remove_tags = [
-            {'class':['thumbnail', 'button']},
+            {'class':['button', 'entry-sharing group', 'wp-paginate',
+                'moving-markup', 'entry-comments']},
 
             ]
-
+    extra_css = '.entry-date { padding-left: 2ex }'
-    recursions = 10
-    match_regexps = [r'/[0-9,]+-(2|3|4|5|6|7|8|9)(,\d+){0,1}.html',r'http://www.time.com/time/specials/packages/article/.*']
 
     preprocess_regexps = [(re.compile(
         r'<meta .+/>'), lambda m:'')]
@@ -45,7 +44,7 @@ class Time(BasicNewsRecipe):
             br.select_form(predicate=lambda f: 'action' in f.attrs and f.attrs['action'] == 'https://auth.time.com/login.php')
             br['username']   = self.username
             br['password'] = self.password
-            br['magcode'] = ['TD']
+            # br['magcode'] = ['TD']
             br.find_control('turl').readonly = False
             br['turl'] = 'http://www.time.com/time/magazine'
             br.find_control('rurl').readonly = False
@@ -104,7 +103,14 @@ class Time(BasicNewsRecipe):
                         method='text').strip()
             if not title: continue
             url = a[0].get('href')
-            url = re.sub('/magazine/article/0,9171','/subscriber/printout/0,8816', url)
+            if url.startswith('/'):
+                url = 'http://www.time.com'+url
+            if '/article/0,' in url:
+                soup = self.index_to_soup(url)
+                a = soup.find('a', href=lambda x:x and '/printout/' in x)
+                url = a['href'].replace('/printout', '/subscriber/printout')
+            else:
+                url += 'print/' if url.endswith('/') else '/print/'
             if url.startswith('/'):
                 url = 'http://www.time.com'+url
             desc = ''
@@ -112,10 +118,18 @@ class Time(BasicNewsRecipe):
             if p:
                 desc = html.tostring(p[0], encoding=unicode,
                         method='text')
-            self.log('\t', title, ':\n\t\t', desc)
+            self.log('\t', title, ':\n\t\t', url)
             yield {
                     'title' : title,
                     'url'   : url,
                     'date'  : '',
                     'description' : desc
                     }
+
+    def preprocess_html(self, soup):
+        for fig in soup.findAll('figure'):
+            img = fig.find('img')
+            if img is not None:
+                fig.replaceWith(img)
+        return soup
+

recipes/tvn24.recipe

@@ -7,19 +7,28 @@ class tvn24(BasicNewsRecipe):
     description   = u'Sport, Biznes, Gospodarka, Informacje, Wiadomości Zawsze aktualne wiadomości z Polski i ze świata'
     category       = 'news'
     language       = 'pl'
-    masthead_url= 'http://www.tvn24.pl/_d/topmenu/logo2.gif'
+    #masthead_url= 'http://www.tvn24.pl/_d/topmenu/logo2.gif'
-    cover_url= 'http://www.tvn24.pl/_d/topmenu/logo2.gif'
+    cover_url= 'http://www.userlogos.org/files/logos/Struna/TVN24.jpg'
-    extra_css= 'ul {list-style: none; padding: 0; margin: 0;} li {float: left;margin: 0 0.15em;}'
+    extra_css = 'ul {list-style:none;} \
+                 li {list-style:none; float: left; margin: 0 0.15em;} \
+                 h2 {font-size: medium} \
+                 .date60m {float: left; margin: 0 10px 0 5px;}'
     remove_empty_feeds = True
     remove_javascript = True
     no_stylesheets = True
-    keep_only_tags=[dict(name='h1', attrs={'class':'standardHeader1'}), dict(attrs={'class':['date60m rd5', 'imageBackground fl rd7', 'contentFromCMS']}), dict(attrs={'class':'mainLeftColumn'})]
+    use_embedded_content = False
-    remove_tags=[dict(attrs={'class':['commentsInfo', 'textSize', 'related newsNews align-right', 'box', 'watchMaterial text']})]
+    ignore_duplicate_articles = {'title', 'url'}
-    #remove_tags_after= dict(attrs={'class':'articleAuthors mb30 mt5 grey_v6'})
+    keep_only_tags=[dict(name='h1', attrs={'class':['size30 mt10 pb10', 'size38 mt10 pb15']}), dict(name='figure', attrs={'class':'articleMainPhoto articleMainPhotoWide'}), dict(name='article', attrs={'class':['mb20', 'mb20 textArticleDefault']}), dict(name='ul', attrs={'class':'newsItem'})]
-    feeds          = [(u'Najnowsze', u'http://www.tvn24.pl/najnowsze.xml'), ]
+    remove_tags = [dict(name='aside', attrs={'class':['innerArticleModule onRight cols externalContent', 'innerArticleModule center']}), dict(name='div', attrs={'class':['thumbsGallery', 'articleTools', 'article right rd7', 'heading', 'quizContent']}), dict(name='a', attrs={'class':'watchMaterial text'}), dict(name='section', attrs={'class':['quiz toCenter', 'quiz toRight']})]
-		#(u'Polska', u'www.tvn24.pl/polska.xml'), (u'\u015awiat', u'http://www.tvn24.pl/swiat.xml'), (u'Sport', u'http://www.tvn24.pl/sport.xml'), (u'Biznes', u'http://www.tvn24.pl/biznes.xml'), (u'Meteo', u'http://www.tvn24.pl/meteo.xml'), (u'Micha\u0142ki', u'http://www.tvn24.pl/michalki.xml'), (u'Kultura', u'http://www.tvn24.pl/kultura.xml')]
+
+    feeds          = [(u'Najnowsze', u'http://www.tvn24.pl/najnowsze.xml'),
+		(u'Polska', u'www.tvn24.pl/polska.xml'), (u'\u015awiat', u'http://www.tvn24.pl/swiat.xml'), (u'Sport', u'http://www.tvn24.pl/sport.xml'), (u'Biznes', u'http://www.tvn24.pl/biznes.xml'), (u'Meteo', u'http://www.tvn24.pl/meteo.xml'), (u'Micha\u0142ki', u'http://www.tvn24.pl/michalki.xml'), (u'Kultura', u'http://www.tvn24.pl/kultura.xml')]
 
     def preprocess_html(self, soup):
         for item in soup.findAll(style=True):
             del item['style']
+        tag = soup.find(name='ul', attrs={'class':'newsItem'})
+        if tag:
+            tag.name='div'
+            tag.li.name='div'
         return soup

recipes/webhosting_pl.recipe

@@ -1,46 +0,0 @@
-from calibre.web.feeds.news import BasicNewsRecipe
-class webhosting_pl(BasicNewsRecipe):
-    title          = u'Webhosting.pl'
-    __author__        = 'fenuks'
-    description   = 'Webhosting.pl to pierwszy na polskim rynku serwis poruszający w szerokim aspekcie tematy związane z hostingiem, globalną Siecią i usługami internetowymi. Głównym celem przedsięwzięcia jest dostarczanie przydatnej i bogatej merytorycznie wiedzy osobom, które chcą tworzyć i efektywnie wykorzystywać współczesny Internet.'
-    category       = 'web'
-    language       = 'pl'
-    cover_url='http://webhosting.pl/images/logo.png'
-    masthead_url='http://webhosting.pl/images/logo.png'
-    oldest_article = 7
-    index='http://webhosting.pl'
-    max_articles_per_feed = 100
-    no_stylesheets = True
-    remove_empty_feeds = True
-    #keep_only_tags= [dict(name='div', attrs={'class':'content_article'}), dict(attrs={'class':'paging'})]
-    #remove_tags=[dict(attrs={'class':['tags', 'wykop', 'facebook_button_count', 'article_bottom']})]
-    feeds          = [(u'Newsy', u'http://webhosting.pl/feed/rss/an'), 
-		(u'Artyku\u0142y', u'http://webhosting.pl/feed/rss/aa'), 
-		(u'Software', u'http://webhosting.pl/feed/rss/n/12'), 
-		(u'Internet', u'http://webhosting.pl/feed/rss/n/9'), 
-		(u'Biznes', u'http://webhosting.pl/feed/rss/n/13'), 
-		(u'Bezpiecze\u0144stwo', u'http://webhosting.pl/feed/rss/n/10'), 
-		(u'Blogi', u'http://webhosting.pl/feed/rss/ab'),  
-		(u'Programowanie', u'http://webhosting.pl/feed/rss/n/8'), 
-		(u'Kursy', u'http://webhosting.pl/feed/rss/n/11'), 
-		(u'Tips&Tricks', u'http://webhosting.pl/feed/rss/n/15'), 
-		(u'Imprezy', u'http://webhosting.pl/feed/rss/n/22'), 
-		(u'Wywiady', u'http://webhosting.pl/feed/rss/n/24'), 
-		(u'Porady', u'http://webhosting.pl/feed/rss/n/3027'), 
-		(u'Znalezione w sieci', u'http://webhosting.pl/feed/rss/n/6804'), 
-		(u'Dev area', u'http://webhosting.pl/feed/rss/n/24504'), 
-		(u"Webmaster's blog", u'http://webhosting.pl/feed/rss/n/29195'), 
-		(u'Domeny', u'http://webhosting.pl/feed/rss/n/11513'), 
-		(u'Praktyka', u'http://webhosting.pl/feed/rss/n/2'), 
-		(u'Serwery', u'http://webhosting.pl/feed/rss/n/11514'), 
-		(u'Inne', u'http://webhosting.pl/feed/rss/n/24811'), 
-		(u'Marketing', u'http://webhosting.pl/feed/rss/n/11535')]
-
-    def print_version(self, url):
-        return url.replace('webhosting.pl', 'webhosting.pl/print')
-
-    def preprocess_html(self, soup):
-        for a in soup('a'):
-            if a.has_key('href') and 'http://' not in a['href'] and 'https://' not in a['href']:
-                a['href']=self.index + a['href']
-        return soup

recipes/yazihane.recipe

@@ -0,0 +1,19 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+import re
+
+class AdvancedUserRecipe1350731826(BasicNewsRecipe):
+    title          = u'Yazihane'
+    oldest_article = 7
+    max_articles_per_feed = 100
+    __author__            = 'A Erdogan'
+    description           = 'Sports Blog'
+    publisher             = 'yazihaneden.com'
+    category              = 'sports, basketball, nba, cycling, euroleague'
+    no_stylesheets        = True
+    use_embedded_content  = False
+    masthead_url          = 'http://www.yazihaneden.com/wp-content/uploads/Untitled-1.png'
+    language = 'tr'
+
+    keep_only_tags = [ dict(name='div', attrs={'id':re.compile('(^|| )post-($|| )', re.DOTALL)})]
+    remove_tags_after = dict(name='div', attrs={'class':'post-footer clear'})
+    feeds          = [(u'Yazihane', u'http://www.yazihaneden.com/feed/')]

recipes/zeitde.recipe

@@ -2,74 +2,52 @@ __license__   = 'GPL v3'
 __copyright__ = '2008, Kovid Goyal <kovid at kovidgoyal.net>'
 
 '''
-Fetch Die Zeit.
+Fetch Zeit-Online.de
 '''
-
 from calibre.web.feeds.news import BasicNewsRecipe
+from datetime import date
 
 class ZeitDe(BasicNewsRecipe):
 
-    title = 'Zeit Online'
+    __author__            = 'Armin Geller'    # AGe 2012-10-13
-    description = 'Zeit Online'
+    title                 = u'Zeit Online'
-    language = 'de'
+    description           = u'German  online portal of newspaper Die Zeit'
-    encoding = 'UTF-8'
+    publisher             = 'ZEIT ONLINE GmbH'
+    category              = 'news, Germany'
+    timefmt               = ' [%a, %d %b %Y]'
+    publication_type      = 'newspaper'
+    language              = 'de_DE'
+    encoding              = 'UTF-8'
 
-    __author__ = 'Martin Pitt, Sujata Raman, Ingo Paschke and Marc Toensing'
+    oldest_article        = 7
-    no_stylesheets = True
+    max_articles_per_feed = 100
+    remove_empty_feeds    = True
+    auto_cleanup          = True
+    # no_stylesheets = True
+    # conversion_options = {'base_font_size': 10}
 
-    max_articles_per_feed = 40
+    masthead_url          = 'http://images.zeit.de/static/img/logo_247x30.png'
 
-    remove_tags = [
+    year = str(date.today().isocalendar()[0]) # [0]=year [1]=week number [2]=week day
-                        dict(name='iframe'),
+    week = str(date.today().isocalendar()[1]+1)
-                        dict(name='div', attrs={'class':["response","pagination block","pagenav","inline link", "copyright"] }),
+    cover_url = 'http://images.zeit.de/bilder/titelseiten_zeit/titelfluss/'  + year + '/0'+ week + '_001.jpg'
-                        dict(name='p', attrs={'class':["ressortbacklink", "copyright"] }),
-                        dict(name='div', attrs={'id':["place_5","place_4","comments"]})
-                      ]
-
-    keep_only_tags = [dict(id=['main'])]
 
     feeds =  [
-               ('Seite 1', 'http://newsfeed.zeit.de/index_xml'),
+               (u'Startseite – Die wichtigsten Themen auf einen Blick', u'http://newsfeed.zeit.de/index_xml'),
-               ('Politik', 'http://newsfeed.zeit.de/politik/index'),
+               (u'Politik – Ausland und Deutschland', u'http://newsfeed.zeit.de/politik/index'),
-               ('Wirtschaft', 'http://newsfeed.zeit.de/wirtschaft/index'),
+               (u'Wirtschaft – Wirtschaft und Unternehmen', u'http://newsfeed.zeit.de/wirtschaft/index'),
-               ('Meinung', 'http://newsfeed.zeit.de/meinung/index'),
+               (u'Meinung – Autoren kommentieren', u'http://newsfeed.zeit.de/meinung/index'),
-               ('Gesellschaft', 'http://newsfeed.zeit.de/gesellschaft/index'),
+               (u'Gesellschaft – Gesellschaft und soziales Leben', u'http://newsfeed.zeit.de/gesellschaft/index'),
-               ('Kultur', 'http://newsfeed.zeit.de/kultur/index'),
+               (u'Kultur – Literatur, Kunst, Film und Musik', u'http://newsfeed.zeit.de/kultur/index'),
-               ('Wissen', 'http://newsfeed.zeit.de/wissen/index'),
+               (u'Wissen – Wissenschaft, Gesundheit, Umwelt und Geschichte', u'http://newsfeed.zeit.de/wissen/index'),
-               ('Digital', 'http://newsfeed.zeit.de/digital/index'),
+               (u'Digital – Hardware, Software, Internet, Datenschutz', u'http://newsfeed.zeit.de/digital/index'),
-               ('Studium', 'http://newsfeed.zeit.de/studium/index'),
+               (u'Studium – ZEIT ONLINE für Studenten', u'http://newsfeed.zeit.de/studium/index'),
-               ('Karriere', 'http://newsfeed.zeit.de/karriere/index'),
+               (u'Karriere – Für Ein-, Um- und Aufsteiger', u'http://newsfeed.zeit.de/karriere/index'),
-               ('Lebensart', 'http://newsfeed.zeit.de/lebensart/index'),
+               (u'Lebensart – Freizeit und Leben', u'http://newsfeed.zeit.de/lebensart/index'),
-               ('Reisen', 'http://newsfeed.zeit.de/reisen/index'),
+               (u'Reisen – All inclusive und individuell', u'http://newsfeed.zeit.de/reisen/index'),
-               ('Auto', 'http://newsfeed.zeit.de/auto/index'),
+               (u'Auto – Modelle und Trends', u'http://newsfeed.zeit.de/auto/index'),
-               ('Sport', 'http://newsfeed.zeit.de/sport/index'),
+               (u'Sport – Sieg und Niederlage', u'http://newsfeed.zeit.de/sport/index')
              ]
 
-    extra_css = '.excerpt{font-size:1em}.reaktion,.taglist,.comments,.reponse,.responsetitle,.responsebody,.reponse,.inline,.date{display:none;}li.date{display:block}'
+    def print_version(self, url):
-
+        return url + '/komplettansicht?print=true'
-    #filter_regexps = [r'ad.de.doubleclick.net/']
-
-    def get_article_url(self, article):
-        ans = article.get('link',None)
-        ans += "?page=all&print=true"
-
-        if 'video' in ans or 'quiz' in ans or 'blog' in ans :
-              ans = None
-        return ans
-
-    def preprocess_html(self, soup):
-        for tag in soup.findAll(name=['ul','li']):
-            tag.name = 'div'
-
-        soup.html['xml:lang'] = self.language.replace('_', '-')
-        soup.html['lang']     = self.language.replace('_', '-')
-        mtag = '<meta http-equiv="Content-Type" content="text/html; charset=' + self.encoding + '">'
-        soup.head.insert(0,mtag)
-        return soup
-
-    def get_cover_url(self):
-        try:
-            inhalt = self.index_to_soup('http://www.zeit.de/inhalt')
-            return inhalt.find('div', attrs={'class':'singlearchive clearfix'}).img['src'].replace('icon_','')
-        except:
-            return 'http://images.zeit.de/bilder/titelseiten_zeit/1946/001_001.jpg'

resources/mime.types

@@ -787,12 +787,10 @@ application/x-font-framemaker
 application/x-font-ghostscript				gsf
 application/x-font-libgrx
 application/x-font-linux-psf				psf
-application/x-font-otf				otf
 application/x-font-pcf				pcf
 application/x-font-snf				snf
 application/x-font-speedo
 application/x-font-sunos-news
-application/x-font-ttf				ttc ttf
 application/x-font-type1				afm pfa pfb pfm
 application/x-font-vfont
 application/x-freemind				mm
@@ -1368,8 +1366,6 @@ text/fb2+xml                               fb2
 text/x-sony-bbeb+xml                       lrs
 application/x-sony-bbeb                    lrf lrx
 application/adobe-page-template+xml        xpgt
-application/x-font-opentype                otf
-application/x-font-truetype                ttf
 application/x-mobipocket-ebook             mobi prc azw
 application/x-topaz-ebook                  tpz azw1
 application/x-mobipocket-subscription      pobi
@@ -1381,3 +1377,7 @@ application/x-cb7                          cb7
 application/x-koboreader-ebook             kobo
 image/wmf                                  wmf
 application/ereader                        pdb
+# See http://idpf.org/epub/30/spec/epub30-publications.html#sec-core-media-types
+application/vnd.ms-opentype                otf
+application/font-woff                      woff
+application/x-font-truetype                ttf

resources/templates/html.css

@@ -236,7 +236,8 @@ th {
 /* inlines */
 
 b, strong {
-  font-weight: bolder;
+  /* ADE doesn't support bolder with embedded fonts */
+  font-weight: bold;
 }
 
 i, cite, em, var, dfn {

session.vim

@@ -8,6 +8,8 @@ let g:syntastic_cpp_include_dirs = [
             \'/usr/include/qt4/QtCore', 
             \'/usr/include/qt4/QtGui', 
             \'/usr/include/qt4',
+            \'/usr/include/freetype2',
+            \'/usr/include/fontconfig',
             \'src/qtcurve/common', 'src/qtcurve',
             \'/usr/include/ImageMagick',
             \]

setup/build_environment.py

@@ -84,6 +84,7 @@ qt_inc = pyqt.qt_inc_dir
 qt_lib = pyqt.qt_lib_dir
 ft_lib_dirs = []
 ft_libs = []
+ft_inc_dirs = []
 jpg_libs = []
 jpg_lib_dirs = []
 fc_inc = '/usr/include/fontconfig'
@@ -94,6 +95,9 @@ chmlib_inc_dirs = chmlib_lib_dirs = []
 sqlite_inc_dirs = []
 icu_inc_dirs = []
 icu_lib_dirs = []
+zlib_inc_dirs = []
+zlib_lib_dirs = []
+zlib_libs = ['z']
 
 if iswindows:
     prefix  = r'C:\cygwin\home\kovid\sw'
@@ -116,6 +120,10 @@ if iswindows:
     jpg_libs = ['jpeg']
     ft_lib_dirs = [sw_lib_dir]
     ft_libs = ['freetype']
+    ft_inc_dirs = [sw_inc_dir]
+    zlib_inc_dirs = [sw_inc_dir]
+    zlib_lib_dirs = [sw_lib_dir]
+    zlib_libs = ['zlib']
 
     magick_inc_dirs = [os.path.join(prefix, 'build', 'ImageMagick-6.7.6')]
     magick_lib_dirs = [os.path.join(magick_inc_dirs[0], 'VisualMagick', 'lib')]
@@ -135,6 +143,8 @@ elif isosx:
     png_inc_dirs = consolidate('PNG_INC_DIR', '/sw/include')
     png_lib_dirs = consolidate('PNG_LIB_DIR', '/sw/lib')
     png_libs = ['png12']
+    ft_libs = ['freetype']
+    ft_inc_dirs = ['/sw/include/freetype2']
 else:
     # Include directories
     png_inc_dirs = pkgconfig_include_dirs('libpng', 'PNG_INC_DIR',
@@ -150,6 +160,10 @@ else:
     if not magick_libs:
         magick_libs = ['MagickWand', 'MagickCore']
     png_libs = ['png']
+    ft_inc_dirs = pkgconfig_include_dirs('freetype2', 'FT_INC_DIR',
+            '/usr/include/freetype2')
+    ft_lib_dirs = pkgconfig_lib_dirs('freetype2', 'FT_LIB_DIR', '/usr/lib')
+    ft_libs = pkgconfig_libs('freetype2', '', '')
 
 
 fc_inc = os.environ.get('FC_INC_DIR', fc_inc)

setup/extensions.py

@@ -17,7 +17,8 @@ from setup.build_environment import (fc_inc, fc_lib, chmlib_inc_dirs, fc_error,
         podofo_inc, podofo_lib, podofo_error, pyqt, OSX_SDK, NMAKE, QMAKE,
         msvc, MT, win_inc, win_lib, win_ddk, magick_inc_dirs, magick_lib_dirs,
         magick_libs, chmlib_lib_dirs, sqlite_inc_dirs, icu_inc_dirs,
-        icu_lib_dirs, win_ddk_lib_dirs)
+        icu_lib_dirs, win_ddk_lib_dirs, ft_libs, ft_lib_dirs, ft_inc_dirs,
+        zlib_libs, zlib_lib_dirs, zlib_inc_dirs)
 MT
 isunix = islinux or isosx or isbsd
 
@@ -50,10 +51,6 @@ class Extension(object):
 reflow_sources = glob.glob(os.path.join(SRC, 'calibre', 'ebooks', 'pdf', '*.cpp'))
 reflow_headers = glob.glob(os.path.join(SRC, 'calibre', 'ebooks', 'pdf', '*.h'))
 
-pdfreflow_libs = []
-if iswindows:
-    pdfreflow_libs = ['advapi32', 'User32', 'Gdi32', 'zlib']
-
 icu_libs = ['icudata', 'icui18n', 'icuuc', 'icuio']
 icu_cflags = []
 if iswindows:
@@ -119,6 +116,12 @@ extensions = [
                     'calibre/utils/lzx/mspack.h'],
             inc_dirs=['calibre/utils/lzx']),
 
+    Extension('freetype',
+        ['calibre/utils/fonts/freetype.cpp'],
+        inc_dirs = ft_inc_dirs,
+        libraries=ft_libs,
+        lib_dirs=ft_lib_dirs),
+
     Extension('fontconfig',
         ['calibre/utils/fonts/fontconfig.c'],
         inc_dirs = [fc_inc],
@@ -126,6 +129,18 @@ extensions = [
         lib_dirs=[fc_lib],
         error=fc_error),
 
+    Extension('woff',
+        ['calibre/utils/fonts/woff/main.c',
+         'calibre/utils/fonts/woff/woff.c'],
+        headers=[
+        'calibre/utils/fonts/woff/woff.h',
+        'calibre/utils/fonts/woff/woff-private.h'],
+        libraries=zlib_libs,
+        lib_dirs=zlib_lib_dirs,
+        inc_dirs=zlib_inc_dirs,
+        ),
+
+
     Extension('msdes',
                 ['calibre/utils/msdes/msdesmodule.c',
                         'calibre/utils/msdes/des.c'],

setup/installer/__init__.py

@@ -6,7 +6,7 @@ __license__   = 'GPL v3'
 __copyright__ = '2009, Kovid Goyal <kovid@kovidgoyal.net>'
 __docformat__ = 'restructuredtext en'
 
-import subprocess, tempfile, os, time
+import subprocess, tempfile, os, time, socket
 
 from setup import Command, installer_name
 from setup.build_environment import HOST, PROJECT
@@ -62,7 +62,10 @@ class Push(Command):
                 r'Owner@winxp:/cygdrive/c/Documents\ and\ Settings/Owner/calibre':'winxp',
                 'kovid@ox:calibre':None,
                 r'kovid@win7:/cygdrive/c/Users/kovid/calibre':'Windows 7',
+                'kovid@getafix:calibre-src':None,
                 }.iteritems():
+            if '@getafix:' in host and socket.gethostname() == 'getafix':
+                continue
             if vmname is None or is_vm_running(vmname):
                 rcmd = BASE_RSYNC + EXCLUDES + ['.', host]
                 print '\n\nPushing to:', vmname or host, '\n'

setup/installer/linux/freeze2.py

@@ -16,7 +16,7 @@ SITE_PACKAGES = ['PIL', 'dateutil', 'dns', 'PyQt4', 'mechanize',
         'sip.so', 'BeautifulSoup.py', 'cssutils', 'encutils', 'lxml',
         'sipconfig.py', 'xdg', 'dbus', '_dbus_bindings.so', 'dbus_bindings.py',
         '_dbus_glib_bindings.so', 'netifaces.so', '_psutil_posix.so',
-        '_psutil_linux.so', 'psutil']
+        '_psutil_linux.so', 'psutil', 'cssselect']
 
 QTDIR          = '/usr/lib/qt4'
 QTDLLS         = ('QtCore', 'QtGui', 'QtNetwork', 'QtSvg', 'QtXml', 'QtWebKit', 'QtDBus')

setup/installer/windows/notes.rst

@@ -30,7 +30,7 @@ If there are no windows binaries already compiled for the version of python you
 
 Run the following command to install python dependencies::
 
-    easy_install --always-unzip -U mechanize pyreadline python-dateutil dnspython cssutils clientform pycrypto
+    easy_install --always-unzip -U mechanize pyreadline python-dateutil dnspython cssutils clientform pycrypto cssselect
 
 Install BeautifulSoup 3.0.x manually into site-packages (3.1.x parses broken HTML very poorly)
 

setup/iso_639/ca.po

@@ -12,14 +12,14 @@ msgstr ""
 "Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
 "devel@lists.alioth.debian.org>\n"
 "POT-Creation-Date: 2011-11-25 14:01+0000\n"
-"PO-Revision-Date: 2012-07-23 10:54+0000\n"
+"PO-Revision-Date: 2012-10-21 16:27+0000\n"
-"Last-Translator: jmontane <Unknown>\n"
+"Last-Translator: Ferran Rius <frius64@hotmail.com>\n"
 "Language-Team: Catalan <linux@softcatala.org>\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"X-Launchpad-Export-Date: 2012-07-24 04:52+0000\n"
+"X-Launchpad-Export-Date: 2012-10-22 04:39+0000\n"
-"X-Generator: Launchpad (build 15668)\n"
+"X-Generator: Launchpad (build 16165)\n"
 "Language: ca\n"
 
 #. name for aaa
@@ -4744,7 +4744,7 @@ msgstr "Chachi"
 
 #. name for cbj
 msgid "Ede Cabe"
-msgstr "Ede Cabe"
+msgstr "Ede;Cabe"
 
 #. name for cbk
 msgid "Chavacano"
@@ -9936,7 +9936,7 @@ msgstr "Ibani"
 
 #. name for ica
 msgid "Ede Ica"
-msgstr "Ede Ica"
+msgstr "Ede;Ica"
 
 #. name for ich
 msgid "Etkywan"
@@ -9964,7 +9964,7 @@ msgstr "Idon"
 
 #. name for idd
 msgid "Ede Idaca"
-msgstr "Ede Idaca"
+msgstr "Ede;Idaca"
 
 #. name for ide
 msgid "Idere"
@@ -9972,7 +9972,7 @@ msgstr "Idere"
 
 #. name for idi
 msgid "Idi"
-msgstr ""
+msgstr "Idi"
 
 #. name for ido
 msgid "Ido"
@@ -10032,35 +10032,35 @@ msgstr "Ebira"
 
 #. name for ige
 msgid "Igede"
-msgstr ""
+msgstr "Igede"
 
 #. name for igg
 msgid "Igana"
-msgstr ""
+msgstr "Igana"
 
 #. name for igl
 msgid "Igala"
-msgstr ""
+msgstr "Igala"
 
 #. name for igm
 msgid "Kanggape"
-msgstr ""
+msgstr "Igom"
 
 #. name for ign
 msgid "Ignaciano"
-msgstr ""
+msgstr "Ignaciano"
 
 #. name for igo
 msgid "Isebe"
-msgstr ""
+msgstr "Isebe"
 
 #. name for igs
 msgid "Interglossa"
-msgstr ""
+msgstr "Interglossa"
 
 #. name for igw
 msgid "Igwe"
-msgstr ""
+msgstr "Igwe"
 
 #. name for ihb
 msgid "Iha Based Pidgin"
@@ -10068,175 +10068,175 @@ msgstr "Iha; parla mixta"
 
 #. name for ihi
 msgid "Ihievbe"
-msgstr ""
+msgstr "Ihievbe"
 
 #. name for ihp
 msgid "Iha"
-msgstr ""
+msgstr "Iha"
 
 #. name for iii
 msgid "Yi; Sichuan"
-msgstr ""
+msgstr "Yi; Sichuan"
 
 #. name for ijc
 msgid "Izon"
-msgstr ""
+msgstr "Izon"
 
 #. name for ije
 msgid "Biseni"
-msgstr ""
+msgstr "Biseni"
 
 #. name for ijj
 msgid "Ede Ije"
-msgstr ""
+msgstr "Ede;Ije"
 
 #. name for ijn
 msgid "Kalabari"
-msgstr ""
+msgstr "Kalabari"
 
 #. name for ijs
 msgid "Ijo; Southeast"
-msgstr ""
+msgstr "Izon;Ijo"
 
 #. name for ike
 msgid "Inuktitut; Eastern Canadian"
-msgstr ""
+msgstr "Inuktitut; Canadà oriental"
 
 #. name for iki
 msgid "Iko"
-msgstr ""
+msgstr "Iko"
 
 #. name for ikk
 msgid "Ika"
-msgstr ""
+msgstr "Ika"
 
 #. name for ikl
 msgid "Ikulu"
-msgstr ""
+msgstr "Ikulu"
 
 #. name for iko
 msgid "Olulumo-Ikom"
-msgstr ""
+msgstr "Olulumo-Ikom"
 
 #. name for ikp
 msgid "Ikpeshi"
-msgstr ""
+msgstr "Ikpeshi"
 
 #. name for ikt
 msgid "Inuktitut; Western Canadian"
-msgstr ""
+msgstr "Inuktitut; Canadà occidental"
 
 #. name for iku
 msgid "Inuktitut"
-msgstr "inuktitut"
+msgstr "Inuktitut"
 
 #. name for ikv
 msgid "Iku-Gora-Ankwa"
-msgstr ""
+msgstr "Iku"
 
 #. name for ikw
 msgid "Ikwere"
-msgstr ""
+msgstr "Ikwere"
 
 #. name for ikx
 msgid "Ik"
-msgstr ""
+msgstr "Ik"
 
 #. name for ikz
 msgid "Ikizu"
-msgstr ""
+msgstr "Ikizu"
 
 #. name for ila
 msgid "Ile Ape"
-msgstr ""
+msgstr "Ile Ape"
 
 #. name for ilb
 msgid "Ila"
-msgstr ""
+msgstr "Ila"
 
 #. name for ile
 msgid "Interlingue"
-msgstr ""
+msgstr "Interlingue"
 
 #. name for ilg
 msgid "Garig-Ilgar"
-msgstr ""
+msgstr "Garig-Ilgar"
 
 #. name for ili
 msgid "Ili Turki"
-msgstr ""
+msgstr "Ili turc"
 
 #. name for ilk
 msgid "Ilongot"
-msgstr ""
+msgstr "Ilong"
 
 #. name for ill
 msgid "Iranun"
-msgstr ""
+msgstr "Iranun"
 
 #. name for ilo
 msgid "Iloko"
-msgstr ""
+msgstr "Ilocà"
 
 #. name for ils
 msgid "International Sign"
-msgstr ""
+msgstr "Signes internacional"
 
 #. name for ilu
 msgid "Ili'uun"
-msgstr ""
+msgstr "Iliun"
 
 #. name for ilv
 msgid "Ilue"
-msgstr ""
+msgstr "Ilue"
 
 #. name for ilw
 msgid "Talur"
-msgstr ""
+msgstr "Talur"
 
 #. name for ima
 msgid "Malasar; Mala"
-msgstr ""
+msgstr "Malasar; Mala"
 
 #. name for ime
 msgid "Imeraguen"
-msgstr ""
+msgstr "Imeraguen"
 
 #. name for imi
 msgid "Anamgura"
-msgstr ""
+msgstr "Anamgura"
 
 #. name for iml
 msgid "Miluk"
-msgstr ""
+msgstr "Miluk"
 
 #. name for imn
 msgid "Imonda"
-msgstr ""
+msgstr "Imonda"
 
 #. name for imo
 msgid "Imbongu"
-msgstr ""
+msgstr "Imbongu"
 
 #. name for imr
 msgid "Imroing"
-msgstr ""
+msgstr "Imroing"
 
 #. name for ims
 msgid "Marsian"
-msgstr ""
+msgstr "Marsià"
 
 #. name for imy
 msgid "Milyan"
-msgstr ""
+msgstr "Mili"
 
 #. name for ina
 msgid "Interlingua (International Auxiliary Language Association)"
-msgstr ""
+msgstr "Interlingua"
 
 #. name for inb
 msgid "Inga"
-msgstr ""
+msgstr "Inga"
 
 #. name for ind
 msgid "Indonesian"
@@ -10244,15 +10244,15 @@ msgstr "Indonesi"
 
 #. name for ing
 msgid "Degexit'an"
-msgstr ""
+msgstr "Degexitan"
 
 #. name for inh
 msgid "Ingush"
-msgstr ""
+msgstr "Ingúix"
 
 #. name for inj
 msgid "Inga; Jungle"
-msgstr ""
+msgstr "Inga; Jungla"
 
 #. name for inl
 msgid "Indonesian Sign Language"
@@ -10260,19 +10260,19 @@ msgstr "Llenguatge de signes indonesi"
 
 #. name for inm
 msgid "Minaean"
-msgstr ""
+msgstr "Minaeà"
 
 #. name for inn
 msgid "Isinai"
-msgstr ""
+msgstr "Isinai"
 
 #. name for ino
 msgid "Inoke-Yate"
-msgstr ""
+msgstr "Inoke-Yate"
 
 #. name for inp
 msgid "Iñapari"
-msgstr ""
+msgstr "Iñapari"
 
 #. name for ins
 msgid "Indian Sign Language"
@@ -10280,27 +10280,27 @@ msgstr "Llenguatge de signes indi"
 
 #. name for int
 msgid "Intha"
-msgstr ""
+msgstr "Intha"
 
 #. name for inz
 msgid "Ineseño"
-msgstr ""
+msgstr "Ineseño"
 
 #. name for ior
 msgid "Inor"
-msgstr ""
+msgstr "Inor"
 
 #. name for iou
 msgid "Tuma-Irumu"
-msgstr ""
+msgstr "Tuma-Irumu"
 
 #. name for iow
 msgid "Iowa-Oto"
-msgstr ""
+msgstr "Iowa-Oto"
 
 #. name for ipi
 msgid "Ipili"
-msgstr ""
+msgstr "Ipili"
 
 #. name for ipk
 msgid "Inupiaq"
@@ -10308,59 +10308,59 @@ msgstr "inupiak"
 
 #. name for ipo
 msgid "Ipiko"
-msgstr ""
+msgstr "Ipiko"
 
 #. name for iqu
 msgid "Iquito"
-msgstr ""
+msgstr "Iquito"
 
 #. name for ire
 msgid "Iresim"
-msgstr ""
+msgstr "Iresim"
 
 #. name for irh
 msgid "Irarutu"
-msgstr ""
+msgstr "Irarutu"
 
 #. name for iri
 msgid "Irigwe"
-msgstr ""
+msgstr "Irigwe"
 
 #. name for irk
 msgid "Iraqw"
-msgstr ""
+msgstr "Iraqw"
 
 #. name for irn
 msgid "Irántxe"
-msgstr ""
+msgstr "Iranxe"
 
 #. name for irr
 msgid "Ir"
-msgstr ""
+msgstr "Ir"
 
 #. name for iru
 msgid "Irula"
-msgstr ""
+msgstr "Irula"
 
 #. name for irx
 msgid "Kamberau"
-msgstr ""
+msgstr "Kamberau"
 
 #. name for iry
 msgid "Iraya"
-msgstr ""
+msgstr "Iraya"
 
 #. name for isa
 msgid "Isabi"
-msgstr ""
+msgstr "Isabi"
 
 #. name for isc
 msgid "Isconahua"
-msgstr ""
+msgstr "Isconahua"
 
 #. name for isd
 msgid "Isnag"
-msgstr ""
+msgstr "Isneg"
 
 #. name for ise
 msgid "Italian Sign Language"
@@ -10372,15 +10372,15 @@ msgstr "Llenguatge de signes irlandès"
 
 #. name for ish
 msgid "Esan"
-msgstr ""
+msgstr "Esan"
 
 #. name for isi
 msgid "Nkem-Nkum"
-msgstr ""
+msgstr "Nkem-Nkum"
 
 #. name for isk
 msgid "Ishkashimi"
-msgstr ""
+msgstr "Ishkashimi"
 
 #. name for isl
 msgid "Icelandic"
@@ -10388,15 +10388,15 @@ msgstr "Islandès"
 
 #. name for ism
 msgid "Masimasi"
-msgstr ""
+msgstr "Masimasi"
 
 #. name for isn
 msgid "Isanzu"
-msgstr ""
+msgstr "Isanzu"
 
 #. name for iso
 msgid "Isoko"
-msgstr ""
+msgstr "Isoco"
 
 #. name for isr
 msgid "Israeli Sign Language"
@@ -10404,11 +10404,11 @@ msgstr "Llenguatge de signes israelià"
 
 #. name for ist
 msgid "Istriot"
-msgstr ""
+msgstr "Istri"
 
 #. name for isu
 msgid "Isu (Menchum Division)"
-msgstr ""
+msgstr "Isu (Divisió de Menchum)"
 
 #. name for ita
 msgid "Italian"
@@ -10416,15 +10416,15 @@ msgstr "Italià"
 
 #. name for itb
 msgid "Itneg; Binongan"
-msgstr ""
+msgstr "Itneg; Binongan"
 
 #. name for ite
 msgid "Itene"
-msgstr ""
+msgstr "Itene"
 
 #. name for iti
 msgid "Itneg; Inlaod"
-msgstr ""
+msgstr "Itneg; Inlaod"
 
 #. name for itk
 msgid "Judeo-Italian"
@@ -10432,107 +10432,107 @@ msgstr "Judeo-italià"
 
 #. name for itl
 msgid "Itelmen"
-msgstr ""
+msgstr "Itelmen"
 
 #. name for itm
 msgid "Itu Mbon Uzo"
-msgstr ""
+msgstr "Itu Mbon Uzo"
 
 #. name for ito
 msgid "Itonama"
-msgstr ""
+msgstr "Itonama"
 
 #. name for itr
 msgid "Iteri"
-msgstr ""
+msgstr "Iteri"
 
 #. name for its
 msgid "Isekiri"
-msgstr ""
+msgstr "Isekiri"
 
 #. name for itt
 msgid "Itneg; Maeng"
-msgstr ""
+msgstr "Itneg; Maeng"
 
 #. name for itv
 msgid "Itawit"
-msgstr ""
+msgstr "Itawit"
 
 #. name for itw
 msgid "Ito"
-msgstr ""
+msgstr "Ito"
 
 #. name for itx
 msgid "Itik"
-msgstr ""
+msgstr "Itik"
 
 #. name for ity
 msgid "Itneg; Moyadan"
-msgstr ""
+msgstr "Itneg; Moyadan"
 
 #. name for itz
 msgid "Itzá"
-msgstr ""
+msgstr "Itzà"
 
 #. name for ium
 msgid "Mien; Iu"
-msgstr ""
+msgstr "Mien; Iu"
 
 #. name for ivb
 msgid "Ibatan"
-msgstr ""
+msgstr "Ibatan"
 
 #. name for ivv
 msgid "Ivatan"
-msgstr ""
+msgstr "Ivatan"
 
 #. name for iwk
 msgid "I-Wak"
-msgstr ""
+msgstr "Iwaak"
 
 #. name for iwm
 msgid "Iwam"
-msgstr ""
+msgstr "Iwam"
 
 #. name for iwo
 msgid "Iwur"
-msgstr ""
+msgstr "Iwur"
 
 #. name for iws
 msgid "Iwam; Sepik"
-msgstr ""
+msgstr "Iwam; Sepik"
 
 #. name for ixc
 msgid "Ixcatec"
-msgstr ""
+msgstr "Ixcatec"
 
 #. name for ixl
 msgid "Ixil"
-msgstr ""
+msgstr "Ixil"
 
 #. name for iya
 msgid "Iyayu"
-msgstr ""
+msgstr "Iyayu"
 
 #. name for iyo
 msgid "Mesaka"
-msgstr ""
+msgstr "Mesaka"
 
 #. name for iyx
 msgid "Yaka (Congo)"
-msgstr ""
+msgstr "Yaka (Congo)"
 
 #. name for izh
 msgid "Ingrian"
-msgstr ""
+msgstr "Ingri"
 
 #. name for izi
 msgid "Izi-Ezaa-Ikwo-Mgbo"
-msgstr ""
+msgstr "Izi-Ezaa-Ikwo-Mgbo"
 
 #. name for izr
 msgid "Izere"
-msgstr ""
+msgstr "Izere"
 
 #. name for jaa
 msgid "Jamamadí"
@@ -18960,7 +18960,7 @@ msgstr ""
 
 #. name for nqk
 msgid "Ede Nago; Kura"
-msgstr ""
+msgstr "Ede;Nago Kura"
 
 #. name for nqm
 msgid "Ndom"
@@ -25172,7 +25172,7 @@ msgstr ""
 
 #. name for tis
 msgid "Itneg; Masadiit"
-msgstr ""
+msgstr "Itneg; Masadiit"
 
 #. name for tit
 msgid "Tinigua"
@@ -29604,7 +29604,7 @@ msgstr ""
 
 #. name for yix
 msgid "Yi; Axi"
-msgstr ""
+msgstr "Yi; Axi"
 
 #. name for yiy
 msgid "Yir Yoront"
@@ -29688,7 +29688,7 @@ msgstr ""
 
 #. name for ylo
 msgid "Yi; Naluo"
-msgstr ""
+msgstr "Yi; Naluo"
 
 #. name for ylr
 msgid "Yalarnnga"
@@ -29764,7 +29764,7 @@ msgstr ""
 
 #. name for ymr
 msgid "Malasar"
-msgstr ""
+msgstr "Malasar"
 
 #. name for yms
 msgid "Mysian"
@@ -30104,7 +30104,7 @@ msgstr ""
 
 #. name for ywq
 msgid "Yi; Wuding-Luquan"
-msgstr ""
+msgstr "Yi; Wuding-luqua"
 
 #. name for ywr
 msgid "Yawuru"

setup/iso_639/nl.po

@@ -12,14 +12,14 @@ msgstr ""
 "Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
 "devel@lists.alioth.debian.org>\n"
 "POT-Creation-Date: 2011-11-25 14:01+0000\n"
-"PO-Revision-Date: 2012-02-01 20:12+0000\n"
+"PO-Revision-Date: 2012-10-24 18:16+0000\n"
 "Last-Translator: drMerry <Unknown>\n"
 "Language-Team: Dutch <vertaling@vrijschrift.org>\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"X-Launchpad-Export-Date: 2012-02-02 05:57+0000\n"
+"X-Launchpad-Export-Date: 2012-10-25 05:35+0000\n"
-"X-Generator: Launchpad (build 14738)\n"
+"X-Generator: Launchpad (build 16179)\n"
 "Language: nl\n"
 
 #. name for aaa
@@ -332,7 +332,7 @@ msgstr "Andegerebinha"
 
 #. name for adh
 msgid "Adhola"
-msgstr ""
+msgstr "Adhola"
 
 #. name for adi
 msgid "Adi"
@@ -30436,11 +30436,11 @@ msgstr ""
 
 #. name for zma
 msgid "Manda (Australia)"
-msgstr ""
+msgstr "Manda (Australië)"
 
 #. name for zmb
 msgid "Zimba"
-msgstr ""
+msgstr "Zimba"
 
 #. name for zmc
 msgid "Margany"

setup/iso_639/ru.po

@@ -13,14 +13,14 @@ msgstr ""
 "Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
 "devel@lists.alioth.debian.org>\n"
 "POT-Creation-Date: 2011-11-25 14:01+0000\n"
-"PO-Revision-Date: 2012-06-14 09:06+0000\n"
+"PO-Revision-Date: 2012-10-20 00:57+0000\n"
-"Last-Translator: Eugene Marshal <Unknown>\n"
+"Last-Translator: Ida Leter <iatheia@yandex.ru>\n"
 "Language-Team: Russian <debian-l10n-russian@lists.debian.org>\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"X-Launchpad-Export-Date: 2012-06-15 04:42+0000\n"
+"X-Launchpad-Export-Date: 2012-10-21 04:41+0000\n"
-"X-Generator: Launchpad (build 15414)\n"
+"X-Generator: Launchpad (build 16165)\n"
 "Language: ru\n"
 
 #. name for aaa
@@ -41,7 +41,7 @@ msgstr "Амал"
 
 #. name for aae
 msgid "Albanian; Arbëreshë"
-msgstr ""
+msgstr "Албанский; диалект Арбереши"
 
 #. name for aaf
 msgid "Aranadan"
@@ -53,7 +53,7 @@ msgstr "Амбрак"
 
 #. name for aah
 msgid "Arapesh; Abu'"
-msgstr ""
+msgstr "Арапешей; Абу'"
 
 #. name for aai
 msgid "Arifama-Miniafia"
@@ -77,15 +77,15 @@ msgstr "Анамбе"
 
 #. name for aao
 msgid "Arabic; Algerian Saharan"
-msgstr ""
+msgstr "Арабский; Алжирская Сахара"
 
 #. name for aap
 msgid "Arára; Pará"
-msgstr ""
+msgstr "Арара; Пара"
 
 #. name for aaq
 msgid "Abnaki; Eastern"
-msgstr ""
+msgstr "Абенаки; Восточный"
 
 #. name for aar
 msgid "Afar"
@@ -97,7 +97,7 @@ msgstr "Асакс"
 
 #. name for aat
 msgid "Albanian; Arvanitika"
-msgstr ""
+msgstr "Албанский; Арнаутский диалект"
 
 #. name for aau
 msgid "Abau"
@@ -125,7 +125,7 @@ msgstr "Банкон"
 
 #. name for abc
 msgid "Ayta; Ambala"
-msgstr ""
+msgstr "Айта; Амбала"
 
 #. name for abd
 msgid "Manide"
@@ -133,7 +133,7 @@ msgstr "Мэнайд"
 
 #. name for abe
 msgid "Abnaki; Western"
-msgstr ""
+msgstr "Абенаки; Западный"
 
 #. name for abf
 msgid "Abai Sungai"
@@ -145,7 +145,7 @@ msgstr "Абага"
 
 #. name for abh
 msgid "Arabic; Tajiki"
-msgstr ""
+msgstr "Арабский; Таджи"
 
 #. name for abi
 msgid "Abidji"
@@ -177,7 +177,7 @@ msgstr "Абон"
 
 #. name for abp
 msgid "Ayta; Abellen"
-msgstr ""
+msgstr "Айта; Абенлен"
 
 #. name for abq
 msgid "Abaza"

setup/translations.py

@@ -152,7 +152,7 @@ class Translations(POT): # {{{
                     subprocess.check_call(['msgfmt', '-o', dest, iso639])
             elif locale not in ('en_GB', 'en_CA', 'en_AU', 'si', 'ur', 'sc',
                     'ltg', 'nds', 'te', 'yi', 'fo', 'sq', 'ast', 'ml', 'ku',
-                    'fr_CA', 'him', 'jv', 'ka'):
+                    'fr_CA', 'him', 'jv', 'ka', 'fur', 'ber'):
                 self.warn('No ISO 639 translations for locale:', locale)
 
         self.write_stats()

src/calibre/__init__.py

@@ -329,6 +329,7 @@ def get_parsed_proxy(typ='http', debug=True):
                     ans['port'] = int(ans['port'])
             except:
                 if debug:
+                    import traceback
                     traceback.print_exc()
             else:
                 if debug:
@@ -689,28 +690,28 @@ def remove_bracketed_text(src,
             buf.append(char)
     return u''.join(buf)
 
-if isosx:
-    import glob, shutil
-    fdir = os.path.expanduser('~/.fonts')
-    try:
-        if not os.path.exists(fdir):
-            os.makedirs(fdir)
-        if not os.path.exists(os.path.join(fdir, 'LiberationSans_Regular.ttf')):
-            base = P('fonts/liberation/*.ttf')
-            for f in glob.glob(base):
-                shutil.copy2(f, fdir)
-    except:
-        import traceback
-        traceback.print_exc()
-
 def load_builtin_fonts():
-    import glob
+    # On linux these are loaded by fontconfig which means that
-    from PyQt4.Qt import QFontDatabase
+    # they are available to Qt as well, since Qt uses fontconfig
-    base = P('fonts/liberation/*.ttf')
+    from calibre.utils.fonts import fontconfig
-    for f in glob.glob(base):
+    fontconfig
-        QFontDatabase.addApplicationFont(f)
-    return 'Liberation Serif', 'Liberation Sans', 'Liberation Mono'
 
+    families = {u'Liberation Serif', u'Liberation Sans', u'Liberation Mono'}
+
+    if iswindows or isosx:
+        import glob
+        from PyQt4.Qt import QFontDatabase
+        families = set()
+        for f in glob.glob(P('fonts/liberation/*.ttf')):
+            with open(f, 'rb') as s:
+                # Windows requires font files to be executable for them to be
+                # loaded successfully, so we use the in memory loader
+                fid = QFontDatabase.addApplicationFontFromData(s.read())
+            if fid > -1:
+                families |= set(map(unicode,
+                    QFontDatabase.applicationFontFamilies(fid)))
+
+    return families
 
 def ipython(user_ns=None):
     from calibre.utils.ipython import ipython

src/calibre/constants.py

@@ -4,7 +4,7 @@ __license__   = 'GPL v3'
 __copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
 __docformat__ = 'restructuredtext en'
 __appname__   = u'calibre'
-numeric_version = (0, 9, 2)
+numeric_version = (0, 9, 3)
 __version__   = u'.'.join(map(unicode, numeric_version))
 __author__    = u"Kovid Goyal <kovid@kovidgoyal.net>"
 
@@ -89,6 +89,8 @@ class Plugins(collections.Mapping):
                 'chm_extra',
                 'icu',
                 'speedup',
+                'freetype',
+                'woff',
             ]
         if iswindows:
             plugins.extend(['winutil', 'wpd', 'winfonts'])

src/calibre/customize/builtins.py

@@ -668,7 +668,7 @@ from calibre.devices.teclast.driver import (TECLAST_K3, NEWSMY, IPAPYRUS,
 from calibre.devices.sne.driver import SNE
 from calibre.devices.misc import (PALMPRE, AVANT, SWEEX, PDNOVEL,
         GEMEI, VELOCITYMICRO, PDNOVEL_KOBO, LUMIREAD, ALURATEK_COLOR,
-        TREKSTOR, EEEREADER, NEXTBOOK, ADAM, MOOVYBOOK, COBY, EX124G)
+        TREKSTOR, EEEREADER, NEXTBOOK, ADAM, MOOVYBOOK, COBY, EX124G, WAYTEQ)
 from calibre.devices.folder_device.driver import FOLDER_DEVICE_FOR_CONFIG
 from calibre.devices.kobo.driver import KOBO, KOBOTOUCH
 from calibre.devices.bambook.driver import BAMBOOK
@@ -742,7 +742,7 @@ plugins += [
     EEEREADER,
     NEXTBOOK,
     ADAM,
-    MOOVYBOOK, COBY, EX124G,
+    MOOVYBOOK, COBY, EX124G, WAYTEQ,
     ITUNES,
     BOEYE_BEX,
     BOEYE_BDX,

src/calibre/customize/profiles.py

@@ -654,6 +654,17 @@ class KindleDXOutput(OutputProfile):
         return u'%s <br/><span style="color: white">%s</span>' % (', '.join(tags),
                 'ttt '.join(tags)+'ttt ')
 
+class KindlePaperWhiteOutput(KindleOutput):
+
+    name = 'Kindle PaperWhite'
+    short_name = 'kindle_pw'
+    description = _('This profile is intended for the Amazon Kindle PaperWhite')
+
+    # Screen size is a best guess
+    screen_size               = (658, 940)
+    dpi                       = 212.0
+    comic_screen_size = screen_size
+
 class KindleFireOutput(KindleDXOutput):
 
     name = 'Kindle Fire'
@@ -766,6 +777,6 @@ output_profiles = [OutputProfile, SonyReaderOutput, SonyReader300Output,
         SonyReaderLandscapeOutput, KindleDXOutput, IlliadOutput,
         IRexDR1000Output, IRexDR800Output, JetBook5Output, NookOutput,
         BambookOutput, NookColorOutput, PocketBook900Output, GenericEink,
-        GenericEinkLarge, KindleFireOutput]
+        GenericEinkLarge, KindleFireOutput, KindlePaperWhiteOutput]
 
 output_profiles.sort(cmp=lambda x,y:cmp(x.name.lower(), y.name.lower()))

src/calibre/customize/ui.py

@@ -447,7 +447,8 @@ def plugin_for_catalog_format(fmt):
 
 # }}}
 
-def device_plugins(include_disabled=False): # {{{
+# Device plugins {{{
+def device_plugins(include_disabled=False):
     for plugin in _initialized_plugins:
         if isinstance(plugin, DevicePlugin):
             if include_disabled or not is_disabled(plugin):
@@ -456,6 +457,13 @@ def device_plugins(include_disabled=False): # {{{
                             False):
                         plugin.do_delayed_plugin_initialization()
                     yield plugin
+
+def disabled_device_plugins():
+    for plugin in _initialized_plugins:
+        if isinstance(plugin, DevicePlugin):
+            if is_disabled(plugin):
+                if platform in plugin.supported_platforms:
+                    yield plugin
 # }}}
 
 # epub fixers {{{

src/calibre/devices/__init__.py

@@ -55,7 +55,8 @@ def get_connected_device():
             break
     return dev
 
-def debug(ioreg_to_tmp=False, buf=None, plugins=None):
+def debug(ioreg_to_tmp=False, buf=None, plugins=None,
+        disabled_plugins=None):
     '''
     If plugins is None, then this method calls startup and shutdown on the
     device plugins. So if you are using it in a context where startup could
@@ -63,7 +64,7 @@ def debug(ioreg_to_tmp=False, buf=None, plugins=None):
     device plugins as the plugins parameter.
     '''
     import textwrap
-    from calibre.customize.ui import device_plugins
+    from calibre.customize.ui import device_plugins, disabled_device_plugins
     from calibre.debug import print_basic_debug_info
     from calibre.devices.scanner import DeviceScanner, win_pnp_drives
     from calibre.constants import iswindows, isosx
@@ -85,6 +86,9 @@ def debug(ioreg_to_tmp=False, buf=None, plugins=None):
             except:
                 out('Startup failed for device plugin: %s'%d)
 
+    if disabled_plugins is None:
+        disabled_plugins = list(disabled_device_plugins())
+
     try:
         print_basic_debug_info(out=buf)
         s = DeviceScanner()
@@ -113,9 +117,10 @@ def debug(ioreg_to_tmp=False, buf=None, plugins=None):
             ioreg += 'Output from osx_get_usb_drives:\n'+drives+'\n\n'
             ioreg += Device.run_ioreg()
         connected_devices = []
-        out('Available plugins:', textwrap.fill(' '.join([x.__class__.__name__ for x in
+        if disabled_plugins:
-            devplugins])))
+            out('\nDisabled plugins:', textwrap.fill(' '.join([x.__class__.__name__ for x in
-        out(' ')
+                disabled_plugins])))
+            out(' ')
         found_dev = False
         for dev in devplugins:
             if not dev.MANAGES_DEVICE_PRESENCE: continue

src/calibre/devices/android/driver.py

@@ -168,7 +168,7 @@ class ANDROID(USBMS):
             # Xperia
             0x13d3 : { 0x3304 : [0x0001, 0x0002] },
 
-            # CREEL?? Also Nextbook
+            # CREEL?? Also Nextbook and Wayteq
             0x5e3 : { 0x726 : [0x222] },
 
             # ZTE
@@ -212,7 +212,7 @@ class ANDROID(USBMS):
             'VIZIO', 'GOOGLE', 'FREESCAL', 'KOBO_INC', 'LENOVO', 'ROCKCHIP',
             'POCKET', 'ONDA_MID', 'ZENITHIN', 'INGENIC', 'PMID701C', 'PD',
             'PMP5097C', 'MASS', 'NOVO7', 'ZEKI', 'COBY', 'SXZ', 'USB_2.0',
-            'COBY_MID', 'VS', 'AINOL']
+            'COBY_MID', 'VS', 'AINOL', 'TOPWISE']
     WINDOWS_MAIN_MEM = ['ANDROID_PHONE', 'A855', 'A853', 'INC.NEXUS_ONE',
             '__UMS_COMPOSITE', '_MB200', 'MASS_STORAGE', '_-_CARD', 'SGH-I897',
             'GT-I9000', 'FILE-STOR_GADGET', 'SGH-T959_CARD', 'SGH-T959', 'SAMSUNG_ANDROID',
@@ -243,7 +243,7 @@ class ANDROID(USBMS):
             'FILE-CD_GADGET', 'GT-I9001_CARD', 'USB_2.0', 'XT875',
             'UMS_COMPOSITE', 'PRO', '.KOBO_VOX', 'SGH-T989_CARD', 'SGH-I727',
             'USB_FLASH_DRIVER', 'ANDROID', 'MID7042', '7035', 'VIEWPAD_7E',
-            'NOVO7']
+            'NOVO7', 'ADVANCED']
 
     OSX_MAIN_MEM = 'Android Device Main Memory'
 

src/calibre/devices/kindle/driver.py

@@ -285,8 +285,8 @@ class KINDLE(USBMS):
 
 class KINDLE2(KINDLE):
 
-    name           = 'Kindle 2/3/4/Touch Device Interface'
+    name           = 'Kindle 2/3/4/Touch/PaperWhite Device Interface'
-    description    = _('Communicate with the Kindle 2/3/4/Touch eBook reader.')
+    description    = _('Communicate with the Kindle 2/3/4/Touch/PaperWhite eBook reader.')
 
     FORMATS     = ['azw', 'mobi', 'azw3', 'prc', 'azw1', 'tpz', 'azw4', 'pobi', 'pdf', 'txt']
     DELETE_EXTS    = KINDLE.DELETE_EXTS + ['.mbp1', '.mbs', '.sdr', '.han']
@@ -327,7 +327,9 @@ class KINDLE2(KINDLE):
     OPT_APNX           = 0
     OPT_APNX_ACCURATE  = 1
     OPT_APNX_CUST_COL  = 2
-    THUMBNAIL_HEIGHT = 180
+    # x330 on the PaperWhite
+    THUMBNAIL_HEIGHT = 330
+    # x262 on the Touch. Doesn't choke on x330, though.
 
     def formats_to_scan_for(self):
         ans = USBMS.formats_to_scan_for(self) | {'azw3'}

src/calibre/devices/kobo/books.py

@@ -20,7 +20,7 @@ class Book(Book_):
     def __init__(self, prefix, lpath, title=None, authors=None, mime=None, date=None, ContentType=None,
                  thumbnail_name=None, size=0, other=None):
 #        debug_print('Book::__init__ - title=', title)
-        show_debug = title is not None and title.lower().find("magic kingdom") >= 0
+        show_debug = title is not None and title.lower().find("xxxxx") >= 0
         if show_debug:
             debug_print("Book::__init__ - title=", title, 'authors=', authors)
             debug_print("Book::__init__ - other=", other)
@@ -31,8 +31,8 @@ class Book(Book_):
 
         if authors is not None and len(authors) > 0:
             self.authors_from_string(authors)
-        if self.author_sort is None or self.author_sort == "Unknown":
+            if self.author_sort is None or self.author_sort == "Unknown":
-            self.author_sort = author_to_author_sort(self.authors)
+                self.author_sort = author_to_author_sort(authors)
 
         self.mime = mime
 
@@ -58,6 +58,7 @@ class Book(Book_):
                                 self.datetime = time.gmtime()
 
         self.contentID = None
+        self.current_shelves     = []
 
         if thumbnail_name is not None:
             self.thumbnail = ImageWrapper(thumbnail_name)
@@ -250,4 +251,3 @@ class KTCollectionsBookList(CollectionsBookList):
 #        debug_print("KTCollectionsBookList:is_debugging - is_debugging=", is_debugging)
 
         return is_debugging
-

src/calibre/devices/kobo/driver.py

@@ -33,11 +33,11 @@ class KOBO(USBMS):
     gui_name = 'Kobo Reader'
     description = _('Communicate with the Kobo Reader')
     author = 'Timothy Legge and David Forrester'
-    version = (2, 0, 1)
+    version = (2, 0, 2)
 
     dbversion = 0
     fwversion = 0
-    supported_dbversion = 62
+    supported_dbversion = 65
     has_kepubs = False
 
     supported_platforms = ['windows', 'osx', 'linux']
@@ -217,7 +217,7 @@ class KOBO(USBMS):
                                 # print 'update_metadata_item returned true'
                                 changed = True
                         else:
-                             debug_print("    Strange:  The file: ", prefix, lpath, " does mot exist!")
+                            debug_print("    Strange:  The file: ", prefix, lpath, " does mot exist!")
                     if lpath in playlist_map and \
                         playlist_map[lpath] not in bl[idx].device_collections:
                             bl[idx].device_collections = playlist_map.get(lpath,[])
@@ -841,8 +841,16 @@ class KOBO(USBMS):
 
 #        debug_print('Finished update_device_database_collections', collections_attributes)
 
+
+    def get_collections_attributes(self):
+        collections = []
+        opts = self.settings()
+        if opts.extra_customization and len(opts.extra_customization[self.OPT_COLLECTIONS]) > 0:
+            collections = [x.lower().strip() for x in opts.extra_customization[self.OPT_COLLECTIONS].split(',')]
+        return collections
+
     def sync_booklists(self, booklists, end_session=True):
-#        debug_print('KOBO: started sync_booklists')
+        debug_print('KOBO: started sync_booklists')
         paths = self.get_device_paths()
 
         blists = {}
@@ -853,12 +861,7 @@ class KOBO(USBMS):
                     blists[i] = booklists[i]
             except IndexError:
                 pass
-        opts = self.settings()
+        collections = self.get_collections_attributes()
-        if opts.extra_customization:
-            collections = [x.lower().strip() for x in
-                    opts.extra_customization[self.OPT_COLLECTIONS].split(',')]
-        else:
-            collections = []
 
         #debug_print('KOBO: collection fields:', collections)
         for i, blist in blists.items():
@@ -869,7 +872,7 @@ class KOBO(USBMS):
             self.update_device_database_collections(blist, collections, oncard)
 
         USBMS.sync_booklists(self, booklists, end_session=end_session)
-        #debug_print('KOBO: finished sync_booklists')
+        debug_print('KOBO: finished sync_booklists')
 
     def rebuild_collections(self, booklist, oncard):
         collections_attributes = []
@@ -1196,7 +1199,7 @@ class KOBOTOUCH(KOBO):
     author = 'David Forrester'
     description = 'Communicate with the Kobo Touch, Glo and Mini firmware. Based on the existing Kobo driver by %s.' % (KOBO.author)
 
-    supported_dbversion = 62
+    supported_dbversion = 65
     min_supported_dbversion = 53
 
     booklist_class = KTCollectionsBookList
@@ -1282,13 +1285,21 @@ class KOBOTOUCH(KOBO):
 
     # Image file name endings. Made up of: image size, min_dbversion, max_dbversion,
     COVER_FILE_ENDINGS = {
-                          ' - N3_LIBRARY_FULL.parsed':[(355,530),0, 99,],
+                          ' - N3_LIBRARY_FULL.parsed':[(355,473),0, 99,],   # Used for Details screen
-#                          ' - N3_LIBRARY_FULL.parsed':[(600,800),0, 99,],
+                          ' - N3_LIBRARY_GRID.parsed':[(149,198),0, 99,],   # Used for library lists
-                          ' - N3_LIBRARY_GRID.parsed':[(149,233),0, 99,],
+                          ' - N3_LIBRARY_LIST.parsed':[(60,90),0, 53,],
-                          ' - N3_LIBRARY_LIST.parsed':[(60,90),0, 99,],
+#                          ' - N3_LIBRARY_SHELF.parsed': [(40,60),0, 52,],
-                          ' - N3_LIBRARY_SHELF.parsed': [(40,60),0, 52,],
+                          ' - N3_FULL.parsed':[(600,800),0, 99,],           # Used for screensaver, home screen 
-                          ' - N3_FULL.parsed':[(600,800),0, 52,],
                           }
+    #Following are the sizes used with pre2.1.4 firmware
+#    COVER_FILE_ENDINGS = {
+#                          ' - N3_LIBRARY_FULL.parsed':[(355,530),0, 99,],   # Used for Details screen
+##                          ' - N3_LIBRARY_FULL.parsed':[(600,800),0, 99,],
+#                          ' - N3_LIBRARY_GRID.parsed':[(149,233),0, 99,],   # Used for library lists
+#                          ' - N3_LIBRARY_LIST.parsed':[(60,90),0, 53,],
+#                          ' - N3_LIBRARY_SHELF.parsed': [(40,60),0, 52,],
+#                          ' - N3_FULL.parsed':[(600,800),0, 99,],           # Used for screensaver if "Full screen" is checked. 
+#                          }
 
     def initialize(self):
         super(KOBOTOUCH, self).initialize()
@@ -1419,6 +1430,7 @@ class KOBOTOUCH(KOBO):
                         debug_print('KoboTouch:update_booklist - bookshelves=', bookshelves)
                         debug_print('KoboTouch:update_booklist - series="%s"' % bl[idx].series)
                         debug_print('KoboTouch:update_booklist - the book=', bl[idx])
+                        debug_print('KoboTouch:update_booklist - the authors=', bl[idx].authors)
                         debug_print('KoboTouch:update_booklist - application_id=', bl[idx].application_id)
                     bl_cache[lpath] = None
                     if bl[idx].title_sort is not None:
@@ -1435,19 +1447,15 @@ class KOBOTOUCH(KOBO):
                         else:
                             debug_print("    Strange:  The file: ", prefix, lpath, " does not exist!")
                             debug_print("KoboTouch:update_booklist - book size=", bl[idx].size)
-#                    if lpath in playlist_map and \
-#                        playlist_map[lpath] not in bl[idx].device_collections:
-#                            bl[idx].device_collections = playlist_map.get(lpath,[])
-#                            changed = True
-
 
                     if show_debug:
                         debug_print("KoboTouch:update_booklist - ContentID='%s'"%ContentID)
                     bl[idx].contentID = ContentID
 
                     if lpath in playlist_map:
-                        bl[idx].device_collections = playlist_map.get(lpath,[])
+                        bl[idx].device_collections  = playlist_map.get(lpath,[])
                         changed = True
+                    bl[idx].current_shelves = bookshelves
 
                     if show_debug:
                         debug_print('KoboTouch:update_booklist - updated bl[idx].device_collections=', bl[idx].device_collections)
@@ -1478,11 +1486,14 @@ class KOBOTOUCH(KOBO):
                         debug_print('KoboTouch:update_booklist - class:', book.__class__)
 #                        debug_print('    resolution:', book.__class__.__mro__)
                         debug_print("    contentid:'%s'"%book.contentID)
-                        debug_print(book)
+                        debug_print("    title:'%s'"%book.title)
+                        debug_print("    the book:", book)
                         debug_print("    author_sort:'%s'"%book.author_sort)
+                        debug_print("    bookshelves:", bookshelves)
 
                     # print 'Update booklist'
-                    book.device_collections = playlist_map.get(lpath,[])# if lpath in playlist_map else []
+                    book.device_collections  = playlist_map.get(lpath,[])# if lpath in playlist_map else []
+                    book.current_shelves = bookshelves
                     book.contentID = ContentID
 #                    debug_print('KoboTouch:update_booklist - title=', title, 'book.device_collections', book.device_collections)
 
@@ -1521,16 +1532,16 @@ class KOBOTOUCH(KOBO):
             # return bytestrings if the content cannot the decoded as unicode
             connection.text_factory = lambda x: unicode(x, "utf-8", "ignore")
 
-            self.bookshelvelist = self.get_bookshelflist(connection)
-
             cursor = connection.cursor()
 
             cursor.execute('select version from dbversion')
             result = cursor.fetchone()
             self.dbversion = result[0]
-
             debug_print("Database Version=%d"%self.dbversion)
 
+            self.bookshelvelist = self.get_bookshelflist(connection)
+            debug_print("KoboTouch:books - shelf list:", self.bookshelvelist)
+
             opts = self.settings()
             if self.dbversion >= 33:
                 query= ('select Title, Attribution, DateCreated, ContentID, MimeType, ContentType, ' \
@@ -1618,14 +1629,15 @@ class KOBOTOUCH(KOBO):
 
         #print "count found in cache: %d, count of files in metadata: %d, need_sync: %s" % \
         #      (len(bl_cache), len(bl), need_sync)
+        # Bypassing the KOBO sync_booklists as that does things we don't need to do
         if need_sync: #self.count_found_in_bl != len(bl) or need_sync:
             debug_print("KoboTouch:books - about to sync_booklists")
             if oncard == 'cardb':
-                self.sync_booklists((None, None, bl))
+                USBMS.sync_booklists(self, (None, None, bl))
             elif oncard == 'carda':
-                self.sync_booklists((None, bl, None))
+                USBMS.sync_booklists(self, (None, bl, None))
             else:
-                self.sync_booklists((bl, None, None))
+                USBMS.sync_booklists(self, (bl, None, None))
 
         self.report_progress(1.0, _('Getting list of books on device...'))
         debug_print("KoboTouch:books - end - oncard='%s'"%oncard)
@@ -1944,6 +1956,7 @@ class KOBOTOUCH(KOBO):
             if self.supports_bookshelves():
                 debug_print("KoboTouch:update_device_database_collections - managing bookshelves.")
                 if bookshelf_attribute:
+                    debug_print("KoboTouch:update_device_database_collections - bookshelf_attribute=", bookshelf_attribute)
                     for book in booklists:
                         if book.application_id is not None:
 #                            debug_print("KoboTouch:update_device_database_collections - about to remove a book from shelves book.title=%s" % book.title)
@@ -1958,11 +1971,7 @@ class KOBOTOUCH(KOBO):
 
     def rebuild_collections(self, booklist, oncard):
         debug_print("KoboTouch:rebuild_collections")
-        collections_attributes = []
+        collections_attributes = self.get_collections_attributes()
-        opts = self.settings()
-        if opts.extra_customization:
-            collections_attributes = [x.strip() for x in
-                    opts.extra_customization[self.OPT_COLLECTIONS].split(',')]
 
         debug_print('KoboTouch:rebuild_collections: collection fields:', collections_attributes)
         self.update_device_database_collections(booklist, collections_attributes, oncard)
@@ -2087,11 +2096,17 @@ class KOBOTOUCH(KOBO):
     def remove_book_from_device_bookshelves(self, connection, book):
         show_debug = self.is_debugging_title(book.title)# or True
 
+        remove_shelf_list = set(book.current_shelves) - set(book.device_collections)# - set(["Im_Reading", "Read", "Closed"])
+
         if show_debug:
-            debug_print('KoboTouch:remove_book_from_device_bookshelves - book.in_library="%s"'%book.application_id)
+            debug_print('KoboTouch:remove_book_from_device_bookshelves - book.application_id="%s"'%book.application_id)
             debug_print('KoboTouch:remove_book_from_device_bookshelves - book.contentID="%s"'%book.contentID)
             debug_print('KoboTouch:remove_book_from_device_bookshelves - book.device_collections=', book.device_collections)
+            debug_print('KoboTouch:remove_book_from_device_bookshelves - book.current_shelves=', book.current_shelves)
+            debug_print('KoboTouch:remove_book_from_device_bookshelves - remove_shelf_list=', remove_shelf_list)
         
+        if len(remove_shelf_list) == 0:
+            return
 
         query = 'DELETE FROM ShelfContent WHERE ContentId = ?'
 
@@ -2148,14 +2163,14 @@ class KOBOTOUCH(KOBO):
         if not self.supports_bookshelves():
             return bookshelves
 
-        query = 'SELECT Name FROM Shelf'
+        query = 'SELECT Name FROM Shelf WHERE _IsDeleted = "false"'
 
         cursor = connection.cursor()
         cursor.execute(query)
-        # count_bookshelves = 0
+#        count_bookshelves = 0
         for i, row in enumerate(cursor):
             bookshelves.append(row[0])
-            # count_bookshelves = i + 1
+#            count_bookshelves = i + 1
 
         cursor.close()
 #        debug_print("KoboTouch:get_bookshelflist - count bookshelves=" + unicode(count_bookshelves))
@@ -2225,6 +2240,7 @@ class KOBOTOUCH(KOBO):
         connection.commit()
         cursor.close()
 
+        # Update the bookshelf list.
         self.bookshelvelist = self.get_bookshelflist(connection)
 
 #        debug_print("KoboTouch:set_bookshelf - end")

src/calibre/devices/misc.py

@@ -407,4 +407,59 @@ class EX124G(USBMS):
             return 'eBooks'
         return self.EBOOK_DIR_CARD_A
 
+class WAYTEQ(USBMS):
+
+    name           = 'WayteQ device interface'
+    gui_name       = 'WayteQ xBook'
+    description    = _('Communicate with the WayteQ Reader')
+    author         = 'Kovid Goyal'
+    supported_platforms = ['windows', 'osx', 'linux']
+
+    # Ordered list of supported formats
+    FORMATS     = ['epub', 'mobi', 'prc', 'fb2', 'txt', 'pdf', 'html', 'rtf', 'chm', 'djvu', 'doc']
+
+    VENDOR_ID   = [0x05e3]
+    PRODUCT_ID  = [0x0726]
+    BCD         = [0x0222]
+
+    EBOOK_DIR_MAIN = 'Documents'
+    SCAN_FROM_ROOT = True
+
+    VENDOR_NAME = 'ROCKCHIP'
+    WINDOWS_MAIN_MEM = WINDOWS_CARD_A_MEM = 'RK28_SDK_DEMO'
+    SUPPORTS_SUB_DIRS = True
+
+    def get_carda_ebook_dir(self, for_upload=False):
+        if for_upload:
+            return 'Documents'
+        return self.EBOOK_DIR_CARD_A
+
+    def windows_sort_drives(self, drives):
+        if len(drives) < 2: return drives
+        main = drives.get('main', None)
+        carda = drives.get('carda', None)
+        if main and carda:
+            drives['main'] = carda
+            drives['carda'] = main
+        return drives
+
+    def linux_swap_drives(self, drives):
+        if len(drives) < 2 or not drives[1] or not drives[2]: return drives
+        drives = list(drives)
+        t = drives[0]
+        drives[0] = drives[1]
+        drives[1] = t
+        return tuple(drives)
+
+    def osx_sort_names(self, names):
+        if len(names) < 2: return names
+        main = names.get('main', None)
+        card = names.get('carda', None)
+
+        if main is not None and card is not None:
+            names['main'] = card
+            names['carda'] = main
+
+        return names
+
 

src/calibre/devices/mtp/books.py

@@ -10,6 +10,7 @@ __docformat__ = 'restructuredtext en'
 import os
 
 from calibre.devices.interface import BookList as BL
+from calibre.ebooks.metadata import title_sort
 from calibre.ebooks.metadata.book.base import Metadata
 from calibre.ebooks.metadata.book.json_codec import JsonCodec
 from calibre.utils.date import utcnow
@@ -62,6 +63,12 @@ class Book(Metadata):
     def __hash__(self):
         return hash((self.storage_id, self.mtp_relpath))
 
+    @property
+    def title_sorter(self):
+        ans = getattr(self, 'title_sort', None)
+        if not ans or self.is_null('title_sort') or ans == _('Unknown'):
+            ans = ''
+        return ans or title_sort(self.title or '')
 
 class JSONCodec(JsonCodec):
     pass

src/calibre/ebooks/__init__.py

@@ -178,18 +178,41 @@ def normalize(x):
 
 def calibre_cover(title, author_string, series_string=None,
         output_format='jpg', title_size=46, author_size=36, logo_path=None):
+    from calibre.utils.config_base import tweaks
     title = normalize(title)
     author_string = normalize(author_string)
     series_string = normalize(series_string)
     from calibre.utils.magick.draw import create_cover_page, TextLine
-    lines = [TextLine(title, title_size), TextLine(author_string, author_size)]
+    text = title + author_string + (series_string or u'')
+    font_path = tweaks['generate_cover_title_font']
+    if font_path is None:
+        font_path = P('fonts/liberation/LiberationSerif-Bold.ttf')
+
+    from calibre.utils.fonts.utils import get_font_for_text
+    font = open(font_path, 'rb').read()
+    c = get_font_for_text(text, font)
+    cleanup = False
+    if c is not None and c != font:
+        from calibre.ptempfile import PersistentTemporaryFile
+        pt = PersistentTemporaryFile('.ttf')
+        pt.write(c)
+        pt.close()
+        font_path = pt.name
+        cleanup = True
+
+    lines = [TextLine(title, title_size, font_path=font_path),
+            TextLine(author_string, author_size, font_path=font_path)]
     if series_string:
-        lines.append(TextLine(series_string, author_size))
+        lines.append(TextLine(series_string, author_size, font_path=font_path))
     if logo_path is None:
         logo_path = I('library.png')
-    return create_cover_page(lines, logo_path, output_format='jpg',
+    try:
+        return create_cover_page(lines, logo_path, output_format='jpg',
             texture_opacity=0.3, texture_data=I('cover_texture.png',
                 data=True))
+    finally:
+        if cleanup:
+            os.remove(font_path)
 
 UNIT_RE = re.compile(r'^(-*[0-9]*[.]?[0-9]*)\s*(%|em|ex|en|px|mm|cm|in|pt|pc)$')
 

src/calibre/ebooks/conversion/cli.py

@@ -132,7 +132,7 @@ def add_pipeline_options(parser, plumber):
                   _('Options to control the look and feel of the output'),
                   [
                       'base_font_size', 'disable_font_rescaling',
-                      'font_size_mapping',
+                      'font_size_mapping', 'embed_font_family',
                       'line_height', 'minimum_line_height',
                       'linearize_tables',
                       'extra_css', 'filter_css',

src/calibre/ebooks/conversion/plumber.py

@@ -193,6 +193,17 @@ OptionRecommendation(name='line_height',
             )
         ),
 
+OptionRecommendation(name='embed_font_family',
+        recommended_value=None, level=OptionRecommendation.LOW,
+        help=_(
+            'Embed the specified font family into the book. This specifies '
+            'the "base" font used for the book. If the input document '
+            'specifies its own fonts, they may override this base font. '
+            'You can use the filter style information option to remove fonts from the '
+            'input document. Note that font embedding only works '
+            'with some output formats, principally EPUB and AZW3.')
+        ),
+
 OptionRecommendation(name='linearize_tables',
             recommended_value=False, level=OptionRecommendation.LOW,
             help=_('Some badly designed documents use tables to control the '

src/calibre/ebooks/metadata/fb2.py

@@ -379,6 +379,10 @@ def set_metadata(stream, mi, apply_null=False, update_timestamp=False):
 
     stream.seek(0)
     stream.truncate()
+    # Apparently there exists FB2 reading software that chokes on the use of
+    # single quotes in xml declaration. Sigh. See
+    # http://www.mobileread.com/forums/showthread.php?p=2273184#post2273184
+    stream.write(b'<?xml version="1.0" encoding="UTF-8"?>\n')
     stream.write(etree.tostring(root, method='xml', encoding='utf-8',
-        xml_declaration=True))
+        xml_declaration=False))
 

src/calibre/ebooks/mobi/__init__.py

@@ -7,6 +7,7 @@ __copyright__ = '2008, Kovid Goyal <kovid at kovidgoyal.net>'
 class MobiError(Exception):
     pass
 
+# That might be a bit small on the PW, but Amazon/KG 2.5 still uses these values, even when delivered to a PW
 MAX_THUMB_SIZE = 16 * 1024
 MAX_THUMB_DIMEN = (180, 240)
 

src/calibre/ebooks/mobi/reader/headers.py

@@ -172,12 +172,9 @@ class BookHeader(object):
                 self.codec = 'cp1252' if not user_encoding else user_encoding
                 log.warn('Unknown codepage %d. Assuming %s' % (self.codepage,
                     self.codec))
-            # There exists some broken DRM removal tool that removes DRM but
+            # Some KF8 files have header length == 256 (generated by kindlegen
-            # leaves the DRM fields in the header yielding a header size of
+            # 2.7?). See https://bugs.launchpad.net/bugs/1067310
-            # 0xF8. The actual value of max_header_length should be 0xE8 but
+            max_header_length = 0x100
-            # it's changed to accommodate this silly tool. Hopefully that will
-            # not break anything else.
-            max_header_length = 0xF8
 
             if (ident == 'TEXTREAD' or self.length < 0xE4 or
                     self.length > max_header_length or

src/calibre/ebooks/mobi/reader/mobi6.py

@@ -753,13 +753,13 @@ class MobiReader(object):
         processed_records = list(range(offset-1, self.book_header.records +
             offset))
 
-        self.mobi_html = ''
+        self.mobi_html = b''
 
         if self.book_header.compression_type == 'DH':
             huffs = [self.sections[i][0] for i in
-                range(self.book_header.huff_offset,
+                xrange(self.book_header.huff_offset,
                     self.book_header.huff_offset + self.book_header.huff_number)]
-            processed_records += list(range(self.book_header.huff_offset,
+            processed_records += list(xrange(self.book_header.huff_offset,
                 self.book_header.huff_offset + self.book_header.huff_number))
             huff = HuffReader(huffs)
             unpack = huff.unpack

src/calibre/ebooks/mobi/writer8/toc.py

@@ -23,6 +23,7 @@ TEMPLATE = '''
   a {{ text-decoration: none }}
   a:hover {{ color: red }}
   {extra_css}
+  {embed_css}
   </style>
 </head>
 <body id="calibre_generated_inline_toc">
@@ -64,8 +65,16 @@ class TOCAdder(object):
 
         self.log('\tGenerating in-line ToC')
 
+        embed_css = ''
+        s = getattr(oeb, 'store_embed_font_rules', None)
+        if getattr(s, 'body_font_family', None):
+            css = [x.cssText for x in s.rules] + [
+                    'body { font-family: %s }'%s.body_font_family]
+            embed_css = '\n\n'.join(css)
+
         root = etree.fromstring(TEMPLATE.format(xhtmlns=XHTML_NS,
-            title=self.title, extra_css=(opts.extra_css or '')))
+            title=self.title, embed_css=embed_css,
+            extra_css=(opts.extra_css or '')))
         parent = XPath('//h:ul')(root)[0]
         parent.text = '\n\t'
         for child in self.oeb.toc:

src/calibre/ebooks/oeb/base.py

@@ -258,7 +258,7 @@ OPF_MIME       = types_map['.opf']
 PAGE_MAP_MIME  = 'application/oebps-page-map+xml'
 OEB_DOC_MIME   = 'text/x-oeb1-document'
 OEB_CSS_MIME   = 'text/x-oeb1-css'
-OPENTYPE_MIME  = 'application/x-font-opentype'
+OPENTYPE_MIME  = types_map['.otf']
 GIF_MIME       = types_map['.gif']
 JPEG_MIME      = types_map['.jpeg']
 PNG_MIME       = types_map['.png']

src/calibre/ebooks/oeb/iterator/book.py

@@ -22,7 +22,6 @@ from calibre.utils.logging import default_log
 from calibre import (guess_type, prepare_string_for_xml,
         xml_replace_entities)
 from calibre.ebooks.oeb.transforms.cover import CoverManager
-
 from calibre.ebooks.oeb.iterator.spine import (SpineItem, create_indexing_data)
 from calibre.ebooks.oeb.iterator.bookmarks import BookmarksMixin
 
@@ -76,7 +75,8 @@ class EbookIterator(BookmarksMixin):
                     return i
 
     def __enter__(self, processed=False, only_input_plugin=False,
-            run_char_count=True, read_anchor_map=True):
+            run_char_count=True, read_anchor_map=True,
+            extract_embedded_fonts_for_qt=False):
         ''' Convert an ebook file into an exploded OEB book suitable for
         display in viewers/preprocessing etc. '''
 
@@ -174,6 +174,16 @@ class EbookIterator(BookmarksMixin):
 
         self.read_bookmarks()
 
+        if extract_embedded_fonts_for_qt:
+            from calibre.ebooks.oeb.iterator.extract_fonts import extract_fonts
+            try:
+                extract_fonts(self.opf, self.log)
+            except:
+                ol = self.log.filter_level
+                self.log.filter_level = self.log.DEBUG
+                self.log.exception('Failed to extract fonts')
+                self.log.filter_level = ol
+
         return self
 
     def __exit__(self, *args):

src/calibre/ebooks/oeb/iterator/extract_fonts.py

@@ -0,0 +1,110 @@
+#!/usr/bin/env python
+# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
+from __future__ import (unicode_literals, division, absolute_import,
+                        print_function)
+
+__license__   = 'GPL v3'
+__copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
+__docformat__ = 'restructuredtext en'
+
+import re, os, logging
+from functools import partial
+from future_builtins import map
+
+class FamilyMap(dict):
+
+    def __init__(self, log):
+        dict.__init__(self)
+        self.replace_map = {}
+        self.added_fonts = set()
+        self.log = log
+
+    def __call__(self, basedir, match):
+        self.read_font_fule(basedir, match.group())
+        return b''
+
+    def finalize(self):
+        if self.replace_map:
+            self.pat = re.compile(br'(font-family.*?)(' +
+                    b'|'.join([re.escape(x) for x in
+                        self.replace_map.iterkeys()])+b')', re.I)
+
+    def replace_font_families(self, raw):
+        if self.replace_map:
+            def sub(m):
+                k = m.group(2).lower()
+                for q, val in self.replace_map.iteritems():
+                    if q.lower() == k.lower():
+                        return m.group().replace(m.group(2), val)
+                return m.group()
+
+            return self.pat.sub(sub, raw)
+
+    def read_font_fule(self, basedir, css):
+        from PyQt4.Qt import QFontDatabase
+        import cssutils
+        cssutils.log.setLevel(logging.ERROR)
+        try:
+            sheet = cssutils.parseString(css, validate=False)
+        except:
+            return
+        for rule in sheet.cssRules:
+            try:
+                s = rule.style
+                src = s.getProperty('src').propertyValue[0].uri
+                font_family = s.getProperty('font-family').propertyValue[0].value
+            except:
+                continue
+            if not src or not font_family:
+                continue
+            font_file = os.path.normcase(os.path.abspath(os.path.join(basedir,
+                src)))
+            if font_file not in self.added_fonts:
+                self.added_fonts.add(font_file)
+                if os.path.exists(font_file):
+                    with open(font_file, 'rb') as f:
+                        idx = QFontDatabase.addApplicationFontFromData(f.read())
+                    if idx > -1:
+                        family = map(unicode,
+                            QFontDatabase.applicationFontFamilies(idx)).next()
+                        self.log('Extracted embedded font:', family, 'from',
+                                os.path.basename(font_file))
+                        if (family and family != font_family and
+                                family not in self.replace_map):
+                            self.log('Replacing font family value:',
+                                    font_family, 'with', family)
+                            self.replace_map[font_family.encode('utf-8')] = \
+                                    family.encode('utf-8')
+
+def extract_fonts(opf, log):
+    '''
+    Extract embedded fonts from the ebook and add them explicitly to the Qt
+    font database to workaround https://bugs.webkit.org/show_bug.cgi?id=29433
+
+    Only works if the font-face and font-family rules are all contained in the
+    CSS files (Also processing the HTML files would be too much of a
+    performance hit, to do robustly).
+    '''
+    css_files = {}
+    font_family_map = FamilyMap(log)
+    pat = re.compile(br'^\s*@font-face\s*{[^}]+}', re.M)
+
+    for item in opf.manifest:
+        if item.mime_type and item.mime_type.lower() in {
+                'text/css', 'text/x-oeb1-css', 'text/x-oeb-css'}:
+            try:
+                with open(item.path, 'rb') as f:
+                    raw = f.read()
+            except EnvironmentError:
+                continue
+            css_files[item.path] = pat.sub(partial(font_family_map,
+                os.path.dirname(item.path)), raw)
+
+    font_family_map.finalize()
+
+    if font_family_map.added_fonts:
+        for path, raw in css_files.iteritems():
+            with open(path, 'wb') as f:
+                nraw = font_family_map.replace_font_families(raw) or raw
+                f.write(nraw)
+

src/calibre/ebooks/oeb/stylizer.py

@@ -20,16 +20,17 @@ except ImportError:
 from cssutils import (profile as cssprofiles, parseString, parseStyle, log as
         cssutils_log, CSSParser, profiles, replaceUrls)
 from lxml import etree
-from lxml.cssselect import css_to_xpath, ExpressionError, SelectorSyntaxError
+from cssselect import HTMLTranslator
+
 from calibre import force_unicode
 from calibre.ebooks import unit_convert
 from calibre.ebooks.oeb.base import XHTML, XHTML_NS, CSS_MIME, OEB_STYLES
 from calibre.ebooks.oeb.base import XPNSMAP, xpath, urlnormalize
-from calibre.ebooks.cssselect import css_to_xpath_no_case
 
 cssutils_log.setLevel(logging.WARN)
 
 _html_css_stylesheet = None
+css_to_xpath = HTMLTranslator().css_to_xpath
 
 def html_css_stylesheet():
     global _html_css_stylesheet
@@ -96,70 +97,97 @@ DEFAULTS = {'azimuth': 'center', 'background-attachment': 'scroll',
 FONT_SIZE_NAMES = set(['xx-small', 'x-small', 'small', 'medium', 'large',
                        'x-large', 'xx-large'])
 
+def xpath_lower_case(arg):
+    'An ASCII lowercase function for XPath'
+    return ("translate(%s, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', "
+            "'abcdefghijklmnopqrstuvwxyz')")%arg
+is_non_whitespace = re.compile(r'^[^ \t\r\n\f]+$').match
+
+class CaseInsensitiveAttributesTranslator(HTMLTranslator):
+    'Treat class and id CSS selectors case-insensitively'
+
+    def xpath_class(self, class_selector):
+        """Translate a class selector."""
+        x = self.xpath(class_selector.selector)
+        if is_non_whitespace(class_selector.class_name):
+            x.add_condition(
+                "%s and contains(concat(' ', normalize-space(%s), ' '), %s)"
+                % ('@class', xpath_lower_case('@class'), self.xpath_literal(
+                    ' '+class_selector.class_name.lower()+' ')))
+        else:
+            x.add_condition('0')
+        return x
+
+    def xpath_hash(self, id_selector):
+        """Translate an ID selector."""
+        x = self.xpath(id_selector.selector)
+        return self.xpath_attrib_equals(x, xpath_lower_case('@id'),
+                (id_selector.id.lower()))
+
+ci_css_to_xpath = CaseInsensitiveAttributesTranslator().css_to_xpath
+
+NULL_NAMESPACE_REGEX = re.compile(ur'''(name\(\) = ['"])h:''')
+def fix_namespace(raw):
+    '''
+    cssselect uses name() = 'h:p' to select tags for some CSS selectors (e.g.
+    h|p+h|p).
+    However, since for us the XHTML namespace is the default namespace (with no
+    prefix), name() is the same as local-name(). So this is a hack to
+    workaround the problem.
+    '''
+    return NULL_NAMESPACE_REGEX.sub(ur'\1', raw)
 
 class CSSSelector(object):
 
-    LOCAL_NAME_RE = re.compile(r"(?<!local-)name[(][)] *= *'[^:]+:")
+    def __init__(self, css, log=None, namespaces=XPNSMAP):
-
+        self.namespaces = namespaces
-    def __init__(self, css, namespaces=XPNSMAP):
+        self.sel = self.build_selector(css, log)
-        if isinstance(css, unicode):
-            # Workaround for bug in lxml on windows/OS X that causes a massive
-            # memory leak with non ASCII selectors
-            css = css.encode('ascii', 'ignore').decode('ascii')
-        try:
-            path = self.LOCAL_NAME_RE.sub(r"local-name() = '", css_to_xpath(css))
-            self.sel1 = etree.XPath(css_to_xpath(css), namespaces=namespaces)
-        except:
-            self.sel1 = lambda x: []
-        try:
-            path = self.LOCAL_NAME_RE.sub(r"local-name() = '",
-                    css_to_xpath_no_case(css))
-            self.sel2 = etree.XPath(path, namespaces=namespaces)
-        except:
-            self.sel2 = lambda x: []
-        self.sel2_use_logged = False
         self.css = css
+        self.used_ci_sel = False
+
+    def build_selector(self, css, log, func=css_to_xpath):
+        try:
+            return etree.XPath(fix_namespace(func(css)), namespaces=self.namespaces)
+        except:
+            if log is not None:
+                log.exception('Failed to parse CSS selector: %r'%css)
+        return None
 
     def __call__(self, node, log):
+        if self.sel is None:
+            return []
         try:
-            ans = self.sel1(node)
+            ans = self.sel(node)
-        except (AssertionError, ExpressionError, etree.XPathSyntaxError,
+        except:
-                    NameError, # thrown on OS X instead of SelectorSyntaxError
+            log.exception(u'Failed to run CSS selector: %s'%self.css)
-                    SelectorSyntaxError):
             return []
 
         if not ans:
-            try:
+            # Try a case insensitive version
-                ans = self.sel2(node)
+            if not hasattr(self, 'ci_sel'):
-            except:
+                self.ci_sel = self.build_selector(self.css, log, ci_css_to_xpath)
-                return []
+                if self.ci_sel is not None:
-            else:
+                    try:
-                if ans and not self.sel2_use_logged:
+                        ans = self.ci_sel(node)
-                    self.sel2_use_logged = True
+                    except:
-                    log.warn('Interpreting class and tag selectors case'
+                        log.exception(u'Failed to run case-insensitive CSS selector: %s'%self.css)
-                        ' insensitively in the CSS selector: %s'%self.css)
+                        return []
+                    if ans:
+                        if not self.used_ci_sel:
+                            log.warn('Interpreting class and id values '
+                                'case-insensitively in selector: %s'%self.css)
+                        self.used_ci_sel = True
         return ans
 
-
-    def __repr__(self):
-        return '<%s %s for %r>' % (
-            self.__class__.__name__,
-            hex(abs(id(self)))[2:],
-            self.css)
-
 _selector_cache = {}
 
 MIN_SPACE_RE = re.compile(r' *([>~+]) *')
 
-def get_css_selector(raw_selector):
+def get_css_selector(raw_selector, log):
     css = MIN_SPACE_RE.sub(r'\1', raw_selector)
-    if isinstance(css, unicode):
-        # Workaround for bug in lxml on windows/OS X that causes a massive
-        # memory leak with non ASCII selectors
-        css = css.encode('ascii', 'ignore').decode('ascii')
     ans = _selector_cache.get(css, None)
     if ans is None:
-        ans = CSSSelector(css)
+        ans = CSSSelector(css, log)
         _selector_cache[css] = ans
     return ans
 
@@ -272,7 +300,7 @@ class Stylizer(object):
             fl = pseudo_pat.search(text)
             if fl is not None:
                 text = text.replace(fl.group(), '')
-            selector = get_css_selector(text)
+            selector = get_css_selector(text, self.oeb.log)
             matches = selector(tree, self.logger)
             if fl is not None:
                 fl = fl.group(1)

src/calibre/ebooks/oeb/transforms/flatcss.py

@@ -14,9 +14,11 @@ from lxml import etree
 import cssutils
 from cssutils.css import Property
 
+from calibre import guess_type
 from calibre.ebooks.oeb.base import (XHTML, XHTML_NS, CSS_MIME, OEB_STYLES,
         namespace, barename, XPath)
 from calibre.ebooks.oeb.stylizer import Stylizer
+from calibre.utils.filenames import ascii_filename
 
 COLLAPSE = re.compile(r'[ \t\r\n\v]+')
 STRIPNUM = re.compile(r'[-0-9]+$')
@@ -101,6 +103,22 @@ def FontMapper(sbase=None, dbase=None, dkey=None):
     else:
         return NullMapper()
 
+class EmbedFontsCSSRules(object):
+
+    def __init__(self, body_font_family, rules):
+        self.body_font_family, self.rules = body_font_family, rules
+        self.href = None
+
+    def __call__(self, oeb):
+        if not self.body_font_family: return None
+        if not self.href:
+            iid, href = oeb.manifest.generate(u'page_styles', u'page_styles.css')
+            rules = [x.cssText for x in self.rules]
+            rules = u'\n\n'.join(rules)
+            sheet = cssutils.parseString(rules, validate=False)
+            self.href = oeb.manifest.add(iid, href, guess_type(href)[0],
+                    data=sheet).href
+        return self.href
 
 class CSSFlattener(object):
     def __init__(self, fbase=None, fkey=None, lineh=None, unfloat=False,
@@ -144,11 +162,65 @@ class CSSFlattener(object):
                 cssutils.replaceUrls(item.data, item.abshref,
                         ignoreImportRules=True)
 
+        self.body_font_family, self.embed_font_rules = self.get_embed_font_info(
+                self.opts.embed_font_family)
+        # Store for use in output plugins/transforms that generate content,
+        # like the AZW3 output inline ToC.
+        self.oeb.store_embed_font_rules = EmbedFontsCSSRules(self.body_font_family,
+                self.embed_font_rules)
         self.stylize_spine()
         self.sbase = self.baseline_spine() if self.fbase else None
         self.fmap = FontMapper(self.sbase, self.fbase, self.fkey)
         self.flatten_spine()
 
+    def get_embed_font_info(self, family, failure_critical=True):
+        efi = []
+        body_font_family = None
+        if not family:
+            return body_font_family, efi
+        from calibre.utils.fonts import fontconfig
+        from calibre.utils.fonts.utils import (get_font_characteristics,
+                panose_to_css_generic_family, get_font_names)
+        faces = fontconfig.fonts_for_family(family)
+        if not faces or not u'normal' in faces:
+            msg = (u'No embeddable fonts found for family: %r'%self.opts.embed_font_family)
+            if failure_critical:
+                raise ValueError(msg)
+            self.oeb.log.warn(msg)
+            return body_font_family, efi
+
+        for k, v in faces.iteritems():
+            ext, data = v[0::2]
+            weight, is_italic, is_bold, is_regular, fs_type, panose = \
+                get_font_characteristics(data)
+            generic_family = panose_to_css_generic_family(panose)
+            family_name, subfamily_name, full_name = get_font_names(data)
+            if k == u'normal':
+                body_font_family = u"'%s',%s"%(family_name, generic_family)
+                if family_name.lower() != family.lower():
+                    self.oeb.log.warn(u'Failed to find an exact match for font:'
+                            u' %r, using %r instead'%(family, family_name))
+                else:
+                    self.oeb.log(u'Embedding font: %s'%family_name)
+            font = {u'font-family':u'"%s"'%family_name}
+            if is_italic:
+                font[u'font-style'] = u'italic'
+            if is_bold:
+                font[u'font-weight'] = u'bold'
+            fid, href = self.oeb.manifest.generate(id=u'font',
+                href=u'%s.%s'%(ascii_filename(full_name).replace(u' ', u'-'), ext))
+            item = self.oeb.manifest.add(fid, href,
+                    guess_type(full_name+'.'+ext)[0],
+                    data=data)
+            item.unload_data_from_memory()
+            font[u'src'] = u'url(%s)'%item.href
+            rule = '@font-face { %s }'%('; '.join(u'%s:%s'%(k, v) for k, v in
+                font.iteritems()))
+            rule = cssutils.parseString(rule)
+            efi.append(rule)
+
+        return body_font_family, efi
+
     def stylize_spine(self):
         self.stylizers = {}
         profile = self.context.source
@@ -170,6 +242,8 @@ class CSSFlattener(object):
                 bs.extend(['page-break-before: always'])
             if self.context.change_justification != 'original':
                 bs.append('text-align: '+ self.context.change_justification)
+            if self.body_font_family:
+                bs.append(u'font-family: '+self.body_font_family)
             body.set('style', '; '.join(bs))
             stylizer = Stylizer(html, item.href, self.oeb, self.context, profile,
                     user_css=self.context.extra_css,
@@ -450,7 +524,8 @@ class CSSFlattener(object):
             items.sort()
             css = ';\n'.join("%s: %s" % (key, val) for key, val in items)
             css = ('@page {\n%s\n}\n'%css) if items else ''
-            rules = [r.cssText for r in stylizer.font_face_rules]
+            rules = [r.cssText for r in stylizer.font_face_rules +
+                    self.embed_font_rules]
             raw = '\n\n'.join(rules)
             css += '\n\n' + raw
             global_css[css].append(item)

src/calibre/ebooks/oeb/transforms/split.py

@@ -13,7 +13,7 @@ import os, math, functools, collections, re, copy
 
 from lxml.etree import XPath as _XPath
 from lxml import etree
-from lxml.cssselect import CSSSelector
+from cssselect import HTMLTranslator
 
 from calibre.ebooks.oeb.base import (OEB_STYLES, XPNSMAP as NAMESPACES,
         urldefrag, rewrite_links, urlunquote, barename, XHTML, urlnormalize)
@@ -73,6 +73,8 @@ class Split(object):
 
     def find_page_breaks(self, item):
         if self.page_break_selectors is None:
+            from calibre.ebooks.oeb.stylizer import fix_namespace
+            css_to_xpath = HTMLTranslator().css_to_xpath
             self.page_break_selectors = set([])
             stylesheets = [x.data for x in self.oeb.manifest if x.media_type in
                     OEB_STYLES]
@@ -83,7 +85,7 @@ class Split(object):
                     'page-break-after'), 'cssText', '').strip().lower()
                 try:
                     if before and before not in {'avoid', 'auto', 'inherit'}:
-                        self.page_break_selectors.add((CSSSelector(rule.selectorText),
+                        self.page_break_selectors.add((XPath(fix_namespace(css_to_xpath(rule.selectorText))),
                             True))
                         if self.remove_css_pagebreaks:
                             rule.style.removeProperty('page-break-before')
@@ -91,7 +93,7 @@ class Split(object):
                     pass
                 try:
                     if after and after not in {'avoid', 'auto', 'inherit'}:
-                        self.page_break_selectors.add((CSSSelector(rule.selectorText),
+                        self.page_break_selectors.add((XPath(fix_namespace(css_to_xpath(rule.selectorText))),
                             False))
                         if self.remove_css_pagebreaks:
                             rule.style.removeProperty('page-break-after')

src/calibre/ebooks/readability/htmls.py

@@ -64,8 +64,12 @@ def shorten_title(doc):
             if e.text_content():
                 add_match(candidates, e.text_content(), orig)
 
-    for item in ['#title', '#head', '#heading', '.pageTitle', '.news_title', '.title', '.head', '.heading', '.contentheading', '.small_header_red']:
+    from cssselect import HTMLTranslator
-        for e in doc.cssselect(item):
+    css_to_xpath = HTMLTranslator().css_to_xpath
+    for item in ('#title', '#head', '#heading', '.pageTitle', '.news_title',
+            '.title', '.head', '.heading', '.contentheading',
+            '.small_header_red'):
+        for e in doc.xpath(css_to_xpath(item)):
             if e.text:
                 add_match(candidates, e.text, orig)
             if e.text_content():

src/calibre/gui2/__init__.py

@@ -12,7 +12,7 @@ from PyQt4.Qt import (QVariant, QFileInfo, QObject, SIGNAL, QBuffer, Qt,
 
 ORG_NAME = 'KovidsBrain'
 APP_UID  = 'libprs500'
-from calibre import prints
+from calibre import prints, load_builtin_fonts
 from calibre.constants import (islinux, iswindows, isbsd, isfrozen, isosx,
         plugins, config_dir, filesystem_encoding, DEBUG)
 from calibre.utils.config import Config, ConfigProxy, dynamic, JSONConfig
@@ -779,6 +779,7 @@ class Application(QApplication):
         qt_app = self
         self._file_open_paths = []
         self._file_open_lock = RLock()
+        load_builtin_fonts()
         self.setup_styles(force_calibre_style)
 
     if DEBUG:

src/calibre/gui2/actions/device.py

@@ -93,7 +93,8 @@ class ShareConnMenu(QMenu): # {{{
                     get_external_ip())
             try :
                 cs_port = content_server_config().parse().port
-                ip_text = _(' [%s, port %d]')%(listen_on, cs_port)
+                ip_text = _(' [%(ip)s, port %(port)d]')%dict(ip=listen_on,
+                        port=cs_port)
             except:
                 ip_text = ' [%s]'%listen_on
             text = _('Stop Content Server') + ip_text

src/calibre/gui2/convert/__init__.py

@@ -191,6 +191,8 @@ class Widget(QWidget):
         elif isinstance(g, (XPathEdit, RegexEdit)):
             g.edit.editTextChanged.connect(f)
             g.edit.currentIndexChanged.connect(f)
+        elif isinstance(g, FontFamilyChooser):
+            g.family_changed.connect(f)
         else:
             raise Exception('Can\'t connect %s'%type(g))
 

src/calibre/gui2/convert/look_and_feel.py

@@ -32,6 +32,7 @@ class LookAndFeelWidget(Widget, Ui_Form):
         Widget.__init__(self, parent,
                 ['change_justification', 'extra_css', 'base_font_size',
                     'font_size_mapping', 'line_height', 'minimum_line_height',
+                    'embed_font_family',
                     'smarten_punctuation', 'unsmarten_punctuation',
                     'disable_font_rescaling', 'insert_blank_line',
                     'remove_paragraph_spacing',

src/calibre/gui2/convert/look_and_feel.ui

@@ -7,27 +7,53 @@
     <x>0</x>
     <y>0</y>
     <width>655</width>
-    <height>522</height>
+    <height>619</height>
    </rect>
   </property>
   <property name="windowTitle">
    <string>Form</string>
   </property>
   <layout class="QGridLayout" name="gridLayout">
-   <item row="0" column="0">
+   <item row="3" column="4">
-    <widget class="QCheckBox" name="opt_disable_font_rescaling">
+    <widget class="QDoubleSpinBox" name="opt_line_height">
-     <property name="text">
+     <property name="suffix">
-      <string>&amp;Disable font size rescaling</string>
+      <string> pt</string>
+     </property>
+     <property name="decimals">
+      <number>1</number>
      </property>
     </widget>
    </item>
-   <item row="1" column="0">
+   <item row="3" column="3">
-    <widget class="QLabel" name="label_18">
+    <widget class="QLabel" name="label">
      <property name="text">
-      <string>Base &amp;font size:</string>
+      <string>Line &amp;height:</string>
      </property>
      <property name="buddy">
-      <cstring>opt_base_font_size</cstring>
+      <cstring>opt_line_height</cstring>
+     </property>
+    </widget>
+   </item>
+   <item row="3" column="0">
+    <widget class="QLabel" name="label_6">
+     <property name="text">
+      <string>Minimum &amp;line height:</string>
+     </property>
+     <property name="buddy">
+      <cstring>opt_minimum_line_height</cstring>
+     </property>
+    </widget>
+   </item>
+   <item row="3" column="1">
+    <widget class="QDoubleSpinBox" name="opt_minimum_line_height">
+     <property name="suffix">
+      <string> %</string>
+     </property>
+     <property name="decimals">
+      <number>1</number>
+     </property>
+     <property name="maximum">
+      <double>900.000000000000000</double>
      </property>
     </widget>
    </item>
@@ -97,49 +123,6 @@
      </item>
     </layout>
    </item>
-   <item row="3" column="0">
-    <widget class="QLabel" name="label_6">
-     <property name="text">
-      <string>Minimum &amp;line height:</string>
-     </property>
-     <property name="buddy">
-      <cstring>opt_minimum_line_height</cstring>
-     </property>
-    </widget>
-   </item>
-   <item row="3" column="1">
-    <widget class="QDoubleSpinBox" name="opt_minimum_line_height">
-     <property name="suffix">
-      <string> %</string>
-     </property>
-     <property name="decimals">
-      <number>1</number>
-     </property>
-     <property name="maximum">
-      <double>900.000000000000000</double>
-     </property>
-    </widget>
-   </item>
-   <item row="4" column="0">
-    <widget class="QLabel" name="label">
-     <property name="text">
-      <string>Line &amp;height:</string>
-     </property>
-     <property name="buddy">
-      <cstring>opt_line_height</cstring>
-     </property>
-    </widget>
-   </item>
-   <item row="4" column="1">
-    <widget class="QDoubleSpinBox" name="opt_line_height">
-     <property name="suffix">
-      <string> pt</string>
-     </property>
-     <property name="decimals">
-      <number>1</number>
-     </property>
-    </widget>
-   </item>
    <item row="5" column="0">
     <widget class="QLabel" name="label_3">
      <property name="text">
@@ -157,14 +140,14 @@
      </property>
     </widget>
    </item>
-   <item row="6" column="0" colspan="2">
+   <item row="7" column="0" colspan="2">
     <widget class="QCheckBox" name="opt_remove_paragraph_spacing">
      <property name="text">
       <string>Remove &amp;spacing between paragraphs</string>
      </property>
     </widget>
    </item>
-   <item row="6" column="3">
+   <item row="7" column="3">
     <widget class="QLabel" name="label_4">
      <property name="text">
       <string>&amp;Indent size:</string>
@@ -177,7 +160,7 @@
      </property>
     </widget>
    </item>
-   <item row="6" column="4">
+   <item row="7" column="4">
     <widget class="QDoubleSpinBox" name="opt_remove_paragraph_spacing_indent_size">
      <property name="toolTip">
       <string>&lt;p&gt;When calibre removes inter paragraph spacing, it automatically sets a paragraph indent, to ensure that paragraphs can be easily distinguished. This option controls the width of that indent.</string>
@@ -199,85 +182,7 @@
      </property>
     </widget>
    </item>
-   <item row="7" column="0" colspan="2">
+   <item row="12" column="0" colspan="5">
-    <widget class="QCheckBox" name="opt_insert_blank_line">
-     <property name="text">
-      <string>Insert &amp;blank line between paragraphs</string>
-     </property>
-    </widget>
-   </item>
-   <item row="7" column="3">
-    <widget class="QLabel" name="label_7">
-     <property name="text">
-      <string>&amp;Line size:</string>
-     </property>
-     <property name="alignment">
-      <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set>
-     </property>
-     <property name="buddy">
-      <cstring>opt_insert_blank_line_size</cstring>
-     </property>
-    </widget>
-   </item>
-   <item row="7" column="4">
-    <widget class="QDoubleSpinBox" name="opt_insert_blank_line_size">
-     <property name="suffix">
-      <string> em</string>
-     </property>
-     <property name="decimals">
-      <number>1</number>
-     </property>
-    </widget>
-   </item>
-   <item row="8" column="0">
-    <widget class="QLabel" name="label_5">
-     <property name="text">
-      <string>Text &amp;justification:</string>
-     </property>
-     <property name="buddy">
-      <cstring>opt_change_justification</cstring>
-     </property>
-    </widget>
-   </item>
-   <item row="8" column="2" colspan="3">
-    <widget class="QComboBox" name="opt_change_justification"/>
-   </item>
-   <item row="9" column="0">
-    <widget class="QCheckBox" name="opt_smarten_punctuation">
-     <property name="text">
-      <string>Smarten &amp;punctuation</string>
-     </property>
-    </widget>
-   </item>
-   <item row="9" column="1" colspan="4">
-    <widget class="QCheckBox" name="opt_asciiize">
-     <property name="text">
-      <string>&amp;Transliterate unicode characters to ASCII</string>
-     </property>
-    </widget>
-   </item>
-   <item row="10" column="0">
-    <widget class="QCheckBox" name="opt_unsmarten_punctuation">
-     <property name="text">
-      <string>&amp;UnSmarten punctuation</string>
-     </property>
-    </widget>
-   </item>
-   <item row="10" column="1" colspan="2">
-    <widget class="QCheckBox" name="opt_keep_ligatures">
-     <property name="text">
-      <string>Keep &amp;ligatures</string>
-     </property>
-    </widget>
-   </item>
-   <item row="10" column="3">
-    <widget class="QCheckBox" name="opt_linearize_tables">
-     <property name="text">
-      <string>&amp;Linearize tables</string>
-     </property>
-    </widget>
-   </item>
-   <item row="11" column="0" colspan="5">
     <widget class="QTabWidget" name="tabWidget">
      <property name="currentIndex">
       <number>0</number>
@@ -378,10 +283,131 @@
          </item>
         </layout>
        </item>
+       <item row="3" column="0">
+        <spacer name="verticalSpacer">
+         <property name="orientation">
+          <enum>Qt::Vertical</enum>
+         </property>
+         <property name="sizeHint" stdset="0">
+          <size>
+           <width>20</width>
+           <height>40</height>
+          </size>
+         </property>
+        </spacer>
+       </item>
       </layout>
      </widget>
     </widget>
    </item>
+   <item row="8" column="0" colspan="2">
+    <widget class="QCheckBox" name="opt_insert_blank_line">
+     <property name="text">
+      <string>Insert &amp;blank line between paragraphs</string>
+     </property>
+    </widget>
+   </item>
+   <item row="8" column="4">
+    <widget class="QDoubleSpinBox" name="opt_insert_blank_line_size">
+     <property name="suffix">
+      <string> em</string>
+     </property>
+     <property name="decimals">
+      <number>1</number>
+     </property>
+    </widget>
+   </item>
+   <item row="9" column="0">
+    <widget class="QLabel" name="label_5">
+     <property name="text">
+      <string>Text &amp;justification:</string>
+     </property>
+     <property name="buddy">
+      <cstring>opt_change_justification</cstring>
+     </property>
+    </widget>
+   </item>
+   <item row="9" column="2" colspan="3">
+    <widget class="QComboBox" name="opt_change_justification"/>
+   </item>
+   <item row="10" column="0">
+    <widget class="QCheckBox" name="opt_smarten_punctuation">
+     <property name="text">
+      <string>Smarten &amp;punctuation</string>
+     </property>
+    </widget>
+   </item>
+   <item row="10" column="1" colspan="4">
+    <widget class="QCheckBox" name="opt_asciiize">
+     <property name="text">
+      <string>&amp;Transliterate unicode characters to ASCII</string>
+     </property>
+    </widget>
+   </item>
+   <item row="11" column="0">
+    <widget class="QCheckBox" name="opt_unsmarten_punctuation">
+     <property name="text">
+      <string>&amp;UnSmarten punctuation</string>
+     </property>
+    </widget>
+   </item>
+   <item row="11" column="1" colspan="2">
+    <widget class="QCheckBox" name="opt_keep_ligatures">
+     <property name="text">
+      <string>Keep &amp;ligatures</string>
+     </property>
+    </widget>
+   </item>
+   <item row="11" column="3">
+    <widget class="QCheckBox" name="opt_linearize_tables">
+     <property name="text">
+      <string>&amp;Linearize tables</string>
+     </property>
+    </widget>
+   </item>
+   <item row="1" column="0">
+    <widget class="QLabel" name="label_18">
+     <property name="text">
+      <string>Base &amp;font size:</string>
+     </property>
+     <property name="buddy">
+      <cstring>opt_base_font_size</cstring>
+     </property>
+    </widget>
+   </item>
+   <item row="8" column="3">
+    <widget class="QLabel" name="label_7">
+     <property name="text">
+      <string>&amp;Line size:</string>
+     </property>
+     <property name="alignment">
+      <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set>
+     </property>
+     <property name="buddy">
+      <cstring>opt_insert_blank_line_size</cstring>
+     </property>
+    </widget>
+   </item>
+   <item row="6" column="0">
+    <widget class="QLabel" name="label_10">
+     <property name="text">
+      <string>&amp;Embed font family:</string>
+     </property>
+     <property name="buddy">
+      <cstring>opt_embed_font_family</cstring>
+     </property>
+    </widget>
+   </item>
+   <item row="0" column="0" colspan="5">
+    <widget class="QCheckBox" name="opt_disable_font_rescaling">
+     <property name="text">
+      <string>&amp;Disable font size rescaling</string>
+     </property>
+    </widget>
+   </item>
+   <item row="6" column="1" colspan="2">
+    <widget class="FontFamilyChooser" name="opt_embed_font_family"/>
+   </item>
   </layout>
  </widget>
  <customwidgets>
@@ -390,6 +416,11 @@
    <extends>QComboBox</extends>
    <header>widgets.h</header>
   </customwidget>
+  <customwidget>
+   <class>FontFamilyChooser</class>
+   <extends>QWidget</extends>
+   <header>calibre/gui2/font_family_chooser.h</header>
+  </customwidget>
  </customwidgets>
  <resources>
   <include location="../../../../resources/images.qrc"/>

src/calibre/gui2/convert/lrf_output.ui

@@ -205,7 +205,7 @@
  <customwidgets>
   <customwidget>
    <class>FontFamilyChooser</class>
-   <extends>QComboBox</extends>
+   <extends>QWidget</extends>
    <header>calibre/gui2/font_family_chooser.h</header>
   </customwidget>
  </customwidgets>

src/calibre/gui2/convert/regex_builder.py

@@ -211,6 +211,9 @@ class RegexEdit(QWidget, Ui_Edit):
         self.button.clicked.connect(self.builder)
 
     def builder(self):
+        if self.db is None:
+            self.doc_cache = _('Click the Open button below to open a '
+                    'ebook to use for testing.')
         bld = RegexBuilder(self.db, self.book_id, self.edit.text(), self.doc_cache, self)
         if bld.cancelled:
             return

src/calibre/gui2/device.py

@@ -11,7 +11,7 @@ from PyQt4.Qt import (QMenu, QAction, QActionGroup, QIcon, SIGNAL,
                      QDialogButtonBox)
 
 from calibre.customize.ui import (available_input_formats, available_output_formats,
-    device_plugins)
+    device_plugins, disabled_device_plugins)
 from calibre.devices.interface import DevicePlugin
 from calibre.devices.errors import (UserFeedback, OpenFeedback, OpenFailed,
                                     InitialConnectionError)
@@ -130,6 +130,7 @@ class DeviceManager(Thread): # {{{
         self.setDaemon(True)
         # [Device driver, Showing in GUI, Ejected]
         self.devices        = list(device_plugins())
+        self.disabled_device_plugins = list(disabled_device_plugins())
         self.managed_devices = [x for x in self.devices if
                 not x.MANAGES_DEVICE_PRESENCE]
         self.unmanaged_devices = [x for x in self.devices if
@@ -425,7 +426,8 @@ class DeviceManager(Thread): # {{{
 
     def _debug_detection(self):
         from calibre.devices import debug
-        raw = debug(plugins=self.devices)
+        raw = debug(plugins=self.devices,
+                disabled_plugins=self.disabled_device_plugins)
         return raw
 
     def debug_detection(self, done):

src/calibre/gui2/font_family_chooser.py

@@ -8,8 +8,10 @@ __copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
 __docformat__ = 'restructuredtext en'
 
 from PyQt4.Qt import (QFontInfo, QFontMetrics, Qt, QFont, QFontDatabase, QPen,
-        QStyledItemDelegate, QSize, QStyle, QComboBox, QStringListModel,
+        QStyledItemDelegate, QSize, QStyle, QStringListModel, pyqtSignal,
-        QDialog, QVBoxLayout, QApplication, QFontComboBox)
+        QDialog, QVBoxLayout, QApplication, QFontComboBox, QPushButton,
+        QToolButton, QGridLayout, QListView, QWidget, QDialogButtonBox, QIcon,
+        QHBoxLayout, QLabel, QModelIndex)
 
 from calibre.utils.icu import sort_key
 
@@ -55,6 +57,12 @@ def writing_system_for_font(font):
 class FontFamilyDelegate(QStyledItemDelegate):
 
     def sizeHint(self, option, index):
+        try:
+            return self.do_size_hint(option, index)
+        except:
+            return QSize(300, 50)
+
+    def do_size_hint(self, option, index):
         text = index.data(Qt.DisplayRole).toString()
         font = QFont(option.font)
         font.setPointSize(QFontInfo(font).pointSize() * 1.5)
@@ -62,6 +70,15 @@ class FontFamilyDelegate(QStyledItemDelegate):
         return QSize(m.width(text), m.height())
 
     def paint(self, painter, option, index):
+        QStyledItemDelegate.paint(self, painter, option, QModelIndex())
+        painter.save()
+        try:
+            self.do_paint(painter, option, index)
+        except:
+            pass
+        painter.restore()
+
+    def do_paint(self, painter, option, index):
         text = unicode(index.data(Qt.DisplayRole).toString())
         font = QFont(option.font)
         font.setPointSize(QFontInfo(font).pointSize() * 1.5)
@@ -75,10 +92,6 @@ class FontFamilyDelegate(QStyledItemDelegate):
         r = option.rect
 
         if option.state & QStyle.State_Selected:
-            painter.save()
-            painter.setBrush(option.palette.highlight())
-            painter.setPen(Qt.NoPen)
-            painter.drawRect(option.rect)
             painter.setPen(QPen(option.palette.highlightedText(), 0))
 
         if (option.direction == Qt.RightToLeft):
@@ -86,7 +99,6 @@ class FontFamilyDelegate(QStyledItemDelegate):
         else:
             r.setLeft(r.left() + 4)
 
-        old = painter.font()
         painter.setFont(font)
         painter.drawText(r, Qt.AlignVCenter|Qt.AlignLeading|Qt.TextSingleLine, text)
 
@@ -100,15 +112,15 @@ class FontFamilyDelegate(QStyledItemDelegate):
                 r.setLeft(r.left() + w)
             painter.drawText(r, Qt.AlignVCenter|Qt.AlignLeading|Qt.TextSingleLine, sample)
 
-        painter.setFont(old)
+class Typefaces(QWidget):
+    pass
 
-        if (option.state & QStyle.State_Selected):
+class FontFamilyDialog(QDialog):
-            painter.restore()
 
-class FontFamilyChooser(QComboBox):
+    def __init__(self, current_family, parent=None):
-
+        QDialog.__init__(self, parent)
-    def __init__(self, parent=None):
+        self.setWindowTitle(_('Choose font family'))
-        QComboBox.__init__(self, parent)
+        self.setWindowIcon(QIcon(I('font.png')))
         from calibre.utils.fonts import fontconfig
         try:
             self.families = fontconfig.find_font_families()
@@ -124,45 +136,94 @@ class FontFamilyChooser(QComboBox):
         self.families.sort(key=sort_key)
         self.families.insert(0, _('None'))
 
+        self.l = l = QGridLayout()
+        self.setLayout(l)
+        self.view = QListView(self)
         self.m = QStringListModel(self.families)
-        self.setModel(self.m)
+        self.view.setModel(self.m)
         self.d = FontFamilyDelegate(self)
-        self.setItemDelegate(self.d)
+        self.view.setItemDelegate(self.d)
-        self.setCurrentIndex(0)
+        self.view.setCurrentIndex(self.m.index(0))
+        if current_family:
+            for i, val in enumerate(self.families):
+                if icu_lower(val) == icu_lower(current_family):
+                    self.view.setCurrentIndex(self.m.index(i))
+                    break
+        self.view.doubleClicked.connect(self.accept, type=Qt.QueuedConnection)
+        self.view.setSelectionMode(self.view.SingleSelection)
+        self.view.setAlternatingRowColors(True)
 
-    def event(self, e):
+        self.bb = QDialogButtonBox(QDialogButtonBox.Ok|QDialogButtonBox.Cancel)
-        if e.type() == e.Resize:
+        self.bb.accepted.connect(self.accept)
-            view = self.view()
+        self.bb.rejected.connect(self.reject)
-            view.window().setFixedWidth(self.width() * 5/3)
+        self.ml = QLabel(_('Choose a font family from the list below:'))
-        return QComboBox.event(self, e)
 
-    def sizeHint(self):
+        self.faces = Typefaces(self)
-        ans = QComboBox.sizeHint(self)
+
-        ans.setWidth(QFontMetrics(self.font()).width('m'*14))
+        l.addWidget(self.ml, 0, 0, 1, 2)
-        return ans
+        l.addWidget(self.view, 1, 0, 1, 1)
+        l.addWidget(self.faces, 1, 1, 1, 1)
+        l.addWidget(self.bb, 2, 0, 1, 2)
+
+        self.resize(600, 500)
+
+    @property
+    def font_family(self):
+        idx = self.view.currentIndex().row()
+        if idx == 0: return None
+        return self.families[idx]
+
+class FontFamilyChooser(QWidget):
+
+    family_changed = pyqtSignal(object)
+
+    def __init__(self, parent=None):
+        QWidget.__init__(self, parent)
+        self.l = l = QHBoxLayout()
+        self.setLayout(l)
+        self.button = QPushButton(self)
+        self.button.setIcon(QIcon(I('font.png')))
+        l.addWidget(self.button)
+        self.default_text = _('Choose &font family')
+        self.font_family = None
+        self.button.clicked.connect(self.show_chooser)
+        self.clear_button = QToolButton(self)
+        self.clear_button.setIcon(QIcon(I('clear_left.png')))
+        self.clear_button.clicked.connect(self.clear_family)
+        l.addWidget(self.clear_button)
+        self.setToolTip = self.button.setToolTip
+        self.toolTip = self.button.toolTip
+        self.clear_button.setToolTip(_('Clear the font family'))
+
+    def clear_family(self):
+        self.font_family = None
 
     @dynamic_property
     def font_family(self):
         def fget(self):
-            idx=  self.currentIndex()
+            return self._current_family
-            if idx == 0: return None
-            return self.families[idx]
         def fset(self, val):
             if not val:
-                idx = 0
+                val = None
-            try:
+            self._current_family = val
-                idx = self.families.index(type(u'')(val))
+            self.button.setText(val or self.default_text)
-            except ValueError:
+            self.family_changed.emit(val)
-                idx = 0
-            self.setCurrentIndex(idx)
         return property(fget=fget, fset=fset)
 
+    def show_chooser(self):
+        d = FontFamilyDialog(self.font_family, self)
+        if d.exec_() == d.Accepted:
+            self.font_family = d.font_family
 
-if __name__ == '__main__':
+def test():
     app = QApplication([])
+    app
     d = QDialog()
     d.setLayout(QVBoxLayout())
     d.layout().addWidget(FontFamilyChooser(d))
     d.layout().addWidget(QFontComboBox(d))
     d.exec_()
 
+if __name__ == '__main__':
+    test()
+

src/calibre/gui2/library/models.py

@@ -1368,6 +1368,8 @@ class DeviceBooksModel(BooksModel): # {{{
                 return QVariant(authors_to_string(au))
             elif cname == 'size':
                 size = self.db[self.map[row]].size
+                if not isinstance(size, (float, int)):
+                    size = 0
                 return QVariant(human_readable(size))
             elif cname == 'timestamp':
                 dt = self.db[self.map[row]].datetime

src/calibre/gui2/main_window.py

@@ -139,3 +139,5 @@ class MainWindow(QMainWindow):
                     show=True)
         except BaseException:
             pass
+        except:
+            pass

src/calibre/gui2/metadata/basic_widgets.py

@@ -270,7 +270,7 @@ class AuthorsEdit(EditWithComplete):
                 import traceback
                 fname = err.filename if err.filename else 'file'
                 error_dialog(self, _('Permission denied'),
-                        _('Could not open %s. Is it being used by another'
+                        _('Could not open "%s". Is it being used by another'
                         ' program?')%fname, det_msg=traceback.format_exc(),
                         show=True)
                 return False

src/calibre/gui2/preferences/adding.ui

@@ -156,7 +156,7 @@ Author matching is exact.</string>
          <property name="toolTip">
           <string>If set, this option will causes calibre to check if a file
  being auto-added is already in the calibre library.
- If it is, a meesage will pop up asking you whether
+ If it is, a message will pop up asking you whether
  you want to add it anyway.</string>
          </property>
          <property name="text">

src/calibre/gui2/preferences/plugins.py

@@ -29,7 +29,7 @@ class PluginModel(QAbstractItemModel, SearchQueryParser): # {{{
         SearchQueryParser.__init__(self, ['all'])
         self.show_only_user_plugins = show_only_user_plugins
         self.icon = QVariant(QIcon(I('plugins.png')))
-        p = QIcon(self.icon).pixmap(32, 32, QIcon.Disabled, QIcon.On)
+        p = QIcon(self.icon).pixmap(64, 64, QIcon.Disabled, QIcon.On)
         self.disabled_icon = QVariant(QIcon(p))
         self._p = p
         self.populate()
@@ -194,17 +194,20 @@ class PluginModel(QAbstractItemModel, SearchQueryParser): # {{{
                         dict(plugin_type=category, plugins=_('plugins')))
         else:
             plugin = self.index_to_plugin(index)
+            disabled = is_disabled(plugin)
             if role == Qt.DisplayRole:
                 ver = '.'.join(map(str, plugin.version))
                 desc = '\n'.join(textwrap.wrap(plugin.description, 100))
                 ans='%s (%s) %s %s\n%s'%(plugin.name, ver, _('by'), plugin.author, desc)
                 c = plugin_customization(plugin)
-                if c:
+                if c and not disabled:
                     ans += _('\nCustomization: ')+c
+                if disabled:
+                    ans += _('\n\nThis plugin has been disabled')
                 return QVariant(ans)
             if role == Qt.DecorationRole:
-                return self.disabled_icon if is_disabled(plugin) else self.icon
+                return self.disabled_icon if disabled else self.icon
-            if role == Qt.ForegroundRole and is_disabled(plugin):
+            if role == Qt.ForegroundRole and disabled:
                 return QVariant(QBrush(Qt.gray))
             if role == Qt.UserRole:
                 return plugin