sync with Kovid's branch

Changelog.yaml

@@ -19,6 +19,113 @@
 #   new recipes:
 #     - title: 
 
+- version: 0.9.2
+  date: 2012-10-11
+
+  new features:
+    - title: "Wireless driver: Speed up deleting of multiple books"
+
+    - title: "E-book viewer: Add options to hide the scrollbar and show reading position in full screen mode."
+      tickets: [1047450]
+
+    - title: "News download: Add a field to allow recipe authors to tell calibre to remove duplicate articles that a re present in more than one section from the download."
+
+    - title: "Metadata download: Turn off the use of the published date for the earliest edition a book as the published date. The earliest edition was identified via worldcat.org, which has rather poor data, leading to the occasional incorrect result. If you want this feature back, you can turn it on again via Preferences->Metadata download."
+ 
+  bug fixes:
+    - title: "ODT Input: More workarounds for the image positioning markup produced by newer versions of LibreOffice."
+      tickets: [1063207]
+
+    - title: "Metadata download dialog: Fix selected cover being changed when covers are re-sorted after download completes"
+
+    - title: "MTP driver: Ignore errors when getting the driveinfo.calibre file from the device and simply regenerate it"
+
+    - title: "E-book viewer: Use the system locale settings to display the 24/12 hour clock in full screen mode"
+      tickets: [1063209]
+
+    - title: "Content Server: Make OPDS initial page respect the fields to display tweak"
+
+    - title: "Fix regression that caused calibre to not use OPF files when adding books recursively from directories with multiple books per directory"
+
+    - title: "KF8 Output: Fix handling of input documents that have URL unsafe characters in the file names of their images."
+      tickets: [1062477]
+
+    - title: "Fix enumeration type custom column not being merged."
+      tickets: [1061602]
+
+  improved recipes:
+    - Pubblico Giornale
+    - Der Spiegel
+    - Shortlist
+    - FHM UK
+    - Countryfile
+    - Cosmo UK
+    - The Sun UK
+    - NME
+
+  new recipes:
+    - title: PVP Online, Mobile Nations, The Verge and Television Without Pity 
+      author: Krittika Goyal
+
+- version: 0.9.1
+  date: 2012-10-05
+
+  new features:
+    - title: "New driver for the Kobo Touch version 2.0+ firmware and Kobo Glo and Mini. See http://www.mobileread.com/forums/showthread.php?t=192863 for details"
+      tickets: [1024983,1059585]
+
+    - title: "Driver for Motorola Defy XT"
+      tickets: [1061903]
+
+    - title: "Wireless driver: Always use automatic metadata management, regardless of the setting in Preferences->Devices"
+
+    - title: "Sending books by email: Allow sending to multiple email addresses at once separated by commas."
+      tickets: [1052332]
+
+    - title: "KF8 Output: Add the css passed in through the extra css conversion option to the generated inline ToC."
+      tickets: [1052343]
+
+    - title: "Windows: No longer use fontconfig to scan the system for available fonts. Instead use the Windows API. Should fix crashes/instability caused by fonts that fontconfig cannot handle"
+
+    - title: "When editing a blank (undefined) published date in the edit metadata dialog, have the calendar popup jump to the current date instead of the date 1-1-101"
+      tickets: [1058531]
+
+    - title: "FB2 Input: Add support for th, code and strikethrough tags and also rowspan, colspan and align attributes."
+      tickets: [1059351,1058591]
+ 
+  bug fixes:
+    - title: "Get Books: Update Woblink"
+
+    - title: "Position the next selected book better after deleting multiple books from the library view"
+      tickets: [1051135]
+
+    - title: "Allow using the Enter key to select the cover in the metadata download dialog"
+      tickets: [1060472]
+
+    - title: "PDF Output: Handle embedded fonts better on linux"
+
+    - title: "HTML Input: Guess mimetype correctly for references to image files without file extensions."
+      tickets: [1059349]
+
+    - title: "Catalog generation: Workaround for bug in the ICU library on older OS X systems that caused catalog generation to fail when certain non-ascii characters are present in the metadata."
+      tickets: [1057862]
+
+    - title: "Wireless driver: Do not abort if BonJour registration fails, as we can still use broadcast to connect"
+
+    - title: "KF8 Output: Fix invalid output being generated for some files with very large blocks of contiguous non-ascii text"
+
+  improved recipes:
+    - FC Knudde 
+    - Stamgaten
+    - Foreign Policy
+    - Washington Post
+    - Twitch Films
+    - Nature News
+
+  new recipes:
+    - title: IOL News and The New Age 
+      author: Darko Miletic
+
 - version: 0.9.0
   date: 2012-09-28
 

manual/gui.rst

@@ -49,9 +49,9 @@ Add books
 
     1. **Add books from a single directory**: Opens a file chooser dialog and allows you to specify which books in a directory should be added. This action is *context sensitive*, i.e. it depends on which :ref:`catalog <catalogs>` you have selected. If you have selected the :guilabel:`Library`, books will be added to the library. If you have selected the ebook reader device, the books will be uploaded to the device, and so on.
 
-    2. **Add books from directories, including sub-directories (One book per directory, assumes every ebook file is the same book in a different format)**: Allows you to choose a directory. The directory and all its sub-directories are scanned recursively, and any ebooks found are added to the library. |app| assumes that each directory contains a single book. All ebook files in a directory are assumed to be the same book in different formats. This action is the inverse of the :ref:`Save to disk <save_to_disk_multiple>` action, i.e. you can :guilabel:`Save to disk`, delete the books and re-add them with no lost information except for the date.
+    2. **Add books from directories, including sub-directories (One book per directory, assumes every ebook file is the same book in a different format)**: Allows you to choose a directory. The directory and all its sub-directories are scanned recursively, and any ebooks found are added to the library. |app| assumes that each directory contains a single book. All ebook files in a directory are assumed to be the same book in different formats. This action is the inverse of the :ref:`Save to disk <save_to_disk_multiple>` action, i.e. you can :guilabel:`Save to disk`, delete the books and re-add them with no lost information except for the date (this assumes you have not changed any of the setting for the Save to disk action).
 
-    3. **Add books from directories, including sub-directories (Multiple books per directory, assumes every ebook file is a different book)**: Allows you to choose a directory. The directory and all its sub-directories are scanned recursively and any ebooks found are added to the library. |app| assumes that each directory contains many books. All ebook files with the same name in a directory are assumed to be the same book in different formats. Ebooks with different names are added as different books. This action is the inverse of the :ref:`Save to disk <save_to_disk_single>` action, i.e. you can :guilabel:`Save to disk`, delete the books and re-add them with no lost information except for the date.
+    3. **Add books from directories, including sub-directories (Multiple books per directory, assumes every ebook file is a different book)**: Allows you to choose a directory. The directory and all its sub-directories are scanned recursively and any ebooks found are added to the library. |app| assumes that each directory contains many books. All ebook files with the same name in a directory are assumed to be the same book in different formats. Ebooks with different names are added as different books. 
 
     4. **Add empty book. (Book Entry with no formats)**: Allows you to create a blank book record. This can be used to then manually fill out the information about a book that you may not have yet in your collection.
 

manual/template_lang.rst

@@ -271,6 +271,8 @@ The following functions are available in addition to those described in single-f
         ap   : use a 12-hour clock instead of a 24-hour clock, with 'ap' replaced by the localized string for am or pm.
         AP   : use a 12-hour clock instead of a 24-hour clock, with 'AP' replaced by the localized string for AM or PM.
         iso  : the date with time and timezone. Must be the only format present.
+        
+    You might get unexpected results if the date you are formatting contains localized month names, which can happen if you changed the format tweaks to contain MMMM. In this case, instead of using something like ``{pubdate:format_date(yyyy)}``, write the template using template program mode as in ``{:'format_date(raw_field('pubdate'),'yyyy')'}``.
     
     * finish_formatting(val, fmt, prefix, suffix) -- apply the format, prefix, and suffix to a value in the same way as done in a template like ``{series_index:05.2f| - |- }``. This function is provided to ease conversion of complex single-function- or template-program-mode templates to :ref:`general program mode <general_mode>` (see below) to take advantage of GPM template compilation. For example, the following program produces the same output as the above template::
     

recipes/cosmopolitan_uk.recipe

@@ -15,6 +15,7 @@ class AdvancedUserRecipe1306097511(BasicNewsRecipe):
     max_articles_per_feed = 20
     remove_empty_feeds = True
     remove_javascript     = True
+    ignore_duplicate_articles = {'title'}
 
     preprocess_regexps = [
     (re.compile(r'<!-- Begin tmpl module_competition_offer -->.*?<!-- End tmpl module_competition_offer-->', re.IGNORECASE | re.DOTALL), lambda match: '')]

recipes/countryfile.recipe

@@ -1,11 +1,13 @@
 from calibre import browser
 from calibre.web.feeds.news import BasicNewsRecipe
+import re
+
 class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     title          = u'Countryfile.com'
     #cover_url = 'http://www.countryfile.com/sites/default/files/imagecache/160px_wide/cover/2_1.jpg'
     __author__ = 'Dave Asbury'
     description           = 'The official website of Countryfile Magazine'
-    # last updated 9/9//12
+    # last updated 7/10/12
     language = 'en_GB'
     oldest_article = 30
     max_articles_per_feed = 25
@@ -13,12 +15,14 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     no_stylesheets = True
     auto_cleanup = True
     #articles_are_obfuscated = True
+    ignore_duplicate_articles = {'title'}
     def get_cover_url(self):
             soup = self.index_to_soup('http://www.countryfile.com/')
-            cov = soup.find(attrs={'class' : 'imagecache imagecache-160px_wide imagecache-linked imagecache-160px_wide_linked'})
+
+            cov = soup.find(attrs={'width' : '160', 'class' : re.compile('imagecache imagecache-160px_wide')})
             print '******** ',cov,' ***'
             cov2 = str(cov)
-            cov2=cov2[140:223]
+            cov2=cov2[10:101]
             print '******** ',cov2,' ***'
             #cov2='http://www.countryfile.com/sites/default/files/imagecache/160px_wide/cover/1b_0.jpg'
             # try to get cover - if can't get known cover
@@ -40,3 +44,6 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     (u'Country News', u'http://www.countryfile.com/rss/news'),
             (u'Countryside', u'http://www.countryfile.com/rss/countryside'),
             ]
+
+
+

recipes/der_spiegel.recipe

@@ -72,7 +72,7 @@ class DerSpiegel(BasicNewsRecipe):
             for article in section.findNextSiblings(['dd','dt']):
                 if article.name == 'dt':
                     break
-                link = article.find('a')
+                link = article.find('a', href=True)
                 title = self.tag_to_string(link).strip()
                 if title in self.empty_articles:
                     continue

recipes/fc_knudde.recipe

@@ -15,5 +15,6 @@ class AdvancedUserRecipe1347706704(BasicNewsRecipe):
     remove_tags_before = dict(id='title')
     remove_tags_after = dict(attrs={'class':'entry-content rich-content'})
     use_embedded_content = True
+    extra_css = 'img{border:0;padding:0;margin:0;width:100%}'
 
     feeds          = [(u'FC Knudde', u'http://www.nusport.nl/feeds/rss/fc-knudde.rss')]

recipes/fhm_uk.recipe

@@ -1,5 +1,6 @@
 from calibre.web.feeds.news import BasicNewsRecipe
 
+
 class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     title          = u'FHM UK'
     description = 'Good News for Men.'
@@ -7,14 +8,15 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     #   cover_url = 'http://profile.ak.fbcdn.net/hprofile-ak-snc4/373529_38324934806_64930243_n.jpg'
     masthead_url = 'http://www.fhm.com/App_Resources/Images/Site/re-design/logo.gif'
     __author__ = 'Dave Asbury'
-    # last updated 1/7/12
+    # last updated 7/10/12
     language = 'en_GB'
-    oldest_article = 28
+    oldest_article = 31
-    max_articles_per_feed = 8
+    max_articles_per_feed = 15
     remove_empty_feeds = True
     no_stylesheets = True
     #auto_cleanup = True
    # articles_are_obfuscated = True
+
     keep_only_tags = [
                dict(name='h1'),
                dict(name='img',attrs={'id' : 'ctl00_Body_imgMainImage'}),
@@ -28,14 +30,12 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
 
     ]
     feeds          = [
-        (u'Homepage 1',u'http://feed43.com/6655867614547036.xml'),
+                        # repeatable search = </div>{|}<a href="{%}" class="{*}">{%}</a>{|}<p>{*}</p>
-        (u'Homepage 2',u'http://feed43.com/4167731873103110.xml'),
+        (u'Homepage',u'http://rss.feedsportal.com/c/375/f/434908/index.rss'),
-        (u'Homepage 3',u'http://feed43.com/7667138788771570.xml'),
+        (u'Funny',u'http://rss.feedsportal.com/c/375/f/434910/index.rss'),
-        (u'Homepage 4',u'http://feed43.com/6550421522527341.xml'),
+        (u'Girls',u'http://rss.feedsportal.com/c/375/f/434913/index.rss'),
-        (u'Funny - The Very Best Of The Internet',u'http://feed43.com/4538510106331565.xml'),
+]
-        (u'Gaming',u'http://feed43.com/6537162612465672.xml'),
+
-        (u'Girls',u'http://feed43.com/4574262733341068.xml'),# edit link http://feed43.com/feed.html?name=4574262733341068
-                           ]
 
     extra_css = '''
                     h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}

recipes/iol_za.recipe

@@ -0,0 +1,47 @@
+__license__   = 'GPL v3'
+__copyright__ = '2012, Darko Miletic <darko.miletic at gmail.com>'
+'''
+www.iol.co.za/news
+'''
+
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class IOL_za(BasicNewsRecipe):
+    title                 = 'IOL News'
+    __author__            = 'Darko Miletic'
+    description           = "South Africa's Premier Online News Source. Discover the world of IOL, News South Africa, Sport, Business, Financial, World News, Entertainment, Technology, Motoring, Travel, Property, Classifieds and more."
+    publisher             = 'Independent Newspapers (Pty) Limited.'
+    category              = 'news, politics, South Africa'
+    oldest_article        = 2
+    max_articles_per_feed = 200
+    no_stylesheets        = True
+    encoding              = 'utf8'
+    use_embedded_content  = False
+    auto_cleanup          = False    
+    language              = 'en_ZA'
+    remove_empty_feeds    = True
+    publication_type      = 'newsportal'
+    masthead_url          = 'http://www.iol.co.za/polopoly_fs/iol-news5-1.989381!/image/464471284.png_gen/derivatives/absolute/464471284.png'
+    extra_css             = """ 
+                                body{font-family: Arial,Helvetica,sans-serif }
+                                img{display: block}
+                            """
+
+    conversion_options = {
+                          'comment'   : description
+                        , 'tags'      : category
+                        , 'publisher' : publisher
+                        , 'language'  : language
+                        }
+
+    remove_tags = [dict(name=['object','embed','iframe','table','meta','link'])]
+    keep_only_tags = [dict(attrs={'class':['article_headers', 'byline', 'aticle_column']})]
+
+
+    feeds = [
+              (u'News'    , u'http://iol.co.za/cmlink/1.640'                          )
+             ,(u'Business', u'http://www.iol.co.za/cmlink/1.730910'                   )
+             ,(u'Sport'   , u'http://iol.co.za/cmlink/sport-category-rss-1.704'       )
+             ,(u'World'   , u'http://iol.co.za/cmlink/news-world-category-rss-1.653'  )
+             ,(u'Africa'  , u'http://iol.co.za/cmlink/news-africa-category-rss-1.654' )
+            ]

recipes/mobilenations.recipe

@@ -0,0 +1,21 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class HindustanTimes(BasicNewsRecipe):
+    title          = u'Mobile Nations'
+    language       = 'en'
+    __author__     = 'Krittika Goyal'
+    oldest_article = 1 #days
+    max_articles_per_feed = 25
+    #encoding = 'cp1252'
+    use_embedded_content = False
+
+    no_stylesheets = True
+    auto_cleanup = True
+    #auto_cleanup_keep = '//div[@class="story-image shadowbox entry-content-asset"]'
+
+
+    feeds          = [
+('News',
+ 'http://www.mobilenations.com/rss/mb.xml'),
+]
+

recipes/nme.recipe

@@ -4,7 +4,7 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
     title          = u'New Musical Express Magazine'
     description = 'Author D.Asbury. UK Rock & Pop Mag. '
     __author__ = 'Dave Asbury'
-    # last updated 9/6/12
+    # last updated 7/10/12
     remove_empty_feeds = True
     remove_javascript     = True
     no_stylesheets = True
@@ -14,26 +14,24 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
     language = 'en_GB'
 
     def get_cover_url(self):
-          soup = self.index_to_soup('http://www.magazinesdirect.com/categories/mens/tv-and-music/')
+        soup = self.index_to_soup('http://www.nme.com/component/subscribe')
-          cov = soup.find(attrs={'title' : 'NME magazine subscriptions'})
+        cov = soup.find(attrs={'id' : 'magazine_cover'})
-          cov2 = 'http://www.magazinesdirect.com'+cov['src']
+        cov2 = str(cov['src'])
-          print '***cov =  ',cov2,' ***'
+        # print '**** Cov url =*', cover_url,'***'
+        #print '**** Cov url =*','http://www.magazinesdirect.com/article_images/articledir_3138/1569221/1_largelisting.jpg','***'
 
-          cover_url = str(cov2)
-         # print '**** Cov url =*', cover_url,'***'
-          #print '**** Cov url =*','http://www.magazinesdirect.com/article_images/articledir_3138/1569221/1_largelisting.jpg','***'
 
-          br = browser()
+        br = browser()
-          br.set_handle_redirect(False)
+        br.set_handle_redirect(False)
-          try:
+        try:
-               br.open_novisit(cov2)
+            br.open_novisit(cov2)
-               cover_url = str(cov2)
+            cover_url = str(cov2)
-          except:
+        except:
-                    cover_url = 'http://tawanda3000.files.wordpress.com/2011/02/nme-logo.jpg'
+                cover_url = 'http://tawanda3000.files.wordpress.com/2011/02/nme-logo.jpg'
-          return cover_url
+        return cover_url
+
     masthead_url   = 'http://tawanda3000.files.wordpress.com/2011/02/nme-logo.jpg'
 
-
     remove_tags = [
         dict( attrs={'class':'clear_icons'}),
         dict( attrs={'class':'share_links'}),
@@ -61,9 +59,15 @@ class AdvancedUserRecipe1306061239(BasicNewsRecipe):
 
 
     feeds          = [
-    (u'NME News', u'http://feeds2.feedburner.com/nmecom/rss/newsxml'),
+    (u'NME News', u'http://feeds.feedburner.com/nmecom/rss/newsxml?format=xml'),
     #(u'Reviews', u'http://feeds2.feedburner.com/nme/SdML'),
-    (u'Reviews',u'http://feed43.com/4138608576351646.xml'),
+    (u'Reviews',u'http://feed43.com/1817687144061333.xml'),
                     (u'Bloggs',u'http://feed43.com/3326754333186048.xml'),
 
     ]
+    extra_css = '''
+                    h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
+                    h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
+                    p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
+                    body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
+        '''

recipes/noz.recipe

@@ -0,0 +1,29 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class AdvancedUserRecipe1344926684(BasicNewsRecipe):
+    title          = u'Neue Osnabrücker Zeitung'
+    __author__ = 'Krittika Goyal'
+    oldest_article = 7
+    max_articles_per_feed = 100
+    #auto_cleanup = True
+    no_stylesheets         = True
+    use_embedded_content   = False
+    language               = 'de'
+    remove_javascript      = True
+
+    keep_only_tags = [dict(name='h1', attrs={'class':'enlargeable'}), dict(name='h2', attrs={'class':'enlargeable vorspann'}), dict(name='div', attrs={'id':'largePicContainer'}), dict(name='span', attrs={'id':'articletext'})]
+    remove_tags = [dict(name='div', attrs={'id':'retresco-title'}),dict(name='div', attrs={'class':'retresco-item s1 relative'}),dict(name='a', attrs={'class':'medium2 largeSpaceTop icon'})]
+
+    feeds          = [(u'Lokales', u'http://www.noz.de/rss/Lokales'),
+(u'Vermischtes', u'http://www.noz.de/rss/Vermischtes'),
+(u'Politik', u'http://www.noz.de/rss/Politik'),
+(u'Wirtschaft', u'http://www.noz.de/rss/Wirtschaft'),
+(u'Kultur', u'http://www.noz.de/rss/Kultur'),
+(u'Medien', u'http://www.noz.de/rss/Medien'),
+(u'Wissenschaft', u'http://www.noz.de/rss/wissenschaft'),
+(u'Sport', u'http://www.noz.de/rss/Sport'),
+(u'Computer', u'http://www.noz.de/rss/Computer'),
+(u'Musik', u'http://www.noz.de/rss/Musik'),
+(u'Szene', u'http://www.noz.de/rss/Szene'),
+(u'Niedersachsen', u'http://www.noz.de/rss/Niedersachsen'),
+(u'Kino', u'http://www.noz.de/rss/Kino')]

recipes/pubblico_giornale.recipe

@@ -1,10 +1,10 @@
 #!/usr/bin/env  python
 __license__   = 'GPL v3'
-__author__    = 'iusvar'
+__copyright__ = 'iusvar'
 __description__ = 'Pubblico giornale'
 
 '''
-http://pubblicogiornale.it/
+[url]http://pubblicogiornale.it/[/url]
 '''
 
 from calibre.web.feeds.news import BasicNewsRecipe
@@ -16,6 +16,14 @@ class Pubblicogiornale(BasicNewsRecipe):
     publisher = 'PUBBLICO EDIZIONI Srl'
     category = 'News'
     language = 'it'
-    __author__ = 'iusvar'
+    __author__    = 'iusvar'
 
-    feeds = [(u'Pubblico giornale', u'http://pubblicogiornale.it/feed/')]
+    feeds = [
+      (u'Politica',  u'http://pubblicogiornale.it/category/politica/feed/'),
+      (u'Mondo', u'http://pubblicogiornale.it/category/mondo/feed/'),
+      (u'Economia',  u'http://pubblicogiornale.it/category/economia-2/feed/'),
+      (u'Sport', u'http://pubblicogiornale.it/category/sport-2/feed/'),
+      (u'Cultura', u'http://pubblicogiornale.it/category/cultura-2/feed/'),
+      (u'Rete', u'http://pubblicogiornale.it/category/rete/feed/'),
+      (u'Illustrazioni',u'http://pubblicogiornale.it/category/illustrazioni/feed/')
+]

recipes/pvp_online.recipe

@@ -0,0 +1,18 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class AdvancedUserRecipe1344926684(BasicNewsRecipe):
+    title          = u'PVP online'
+    __author__ = 'Krittika Goyal'
+    oldest_article = 7
+    max_articles_per_feed = 100
+    #auto_cleanup = True
+    no_stylesheets         = True
+    use_embedded_content   = False
+    language               = 'en'
+    remove_javascript      = True
+
+    keep_only_tags = [dict(name='div', attrs={'class':'body'})]
+    remove_tags = [dict(name='div', attrs={'class':'prevBg'}),dict(name='div', attrs={'class':'nextBg'}),dict(name='div', attrs={'class':'postMeta'})]
+
+    feeds          = [(u'Comics', u'http://pvponline.com/feed'), ]
+

recipes/shortlist.recipe

@@ -5,13 +5,15 @@ class AdvancedUserRecipe1324663493(BasicNewsRecipe):
     title          = u'Shortlist'
     description = 'Articles From Shortlist.com'
     # I've set oldest article to 7 days as the website updates weekly
-    oldest_article = 7
+    oldest_article = 8
-    max_articles_per_feed = 12
+    max_articles_per_feed = 20
     remove_empty_feeds = True
     remove_javascript     = True
     no_stylesheets = True
+    ignore_duplicate_articles = {'title'}
+
     __author__ = 'Dave Asbury'
-    # last updated 19/5/12
+    # last updated 7/10/12
     language = 'en_GB'
     def get_cover_url(self):
             soup = self.index_to_soup('http://www.shortlist.com')
@@ -45,17 +47,16 @@ class AdvancedUserRecipe1324663493(BasicNewsRecipe):
                      ]
 
     feeds          = [
-                               (u'Home carousel',u'http://feed43.com/7106317222455380.xml'),
+                               #edit http://feed43.com/feed.html?name=3156308700147005
-                               (u'This Weeks Issue', u'http://feed43.com/0323588208751786.xml'),
+                               # repeatable pattern = <h3>{_}<a href="{%}">{%}</a>{*}</h3>
-         (u'Cool Stuff',u'http://feed43.com/6253845228768456.xml'),
+
-                                (u'Style',u'http://feed43.com/7217107577215678.xml'),
+                                (u'This Weeks Issue', u'http://feed43.com/5205766657404804.xml'),
-                                (u'Films',u'http://feed43.com/3101308515277265.xml'),
+                                (u'Home Page',u'http://feed43.com/3156308700147005.xml'),
-         (u'Music',u'http://feed43.com/2416400550560162.xml'),
+                                (u'Cool Stuff',u'http://feed43.com/1557051772026706.xml'),
-         (u'TV',u'http://feed43.com/4781172470717123.xml'),
+                                (u'Style',u'http://feed43.com/4168836374571502.xml'),
-         (u'Sport',u'http://feed43.com/5303151885853308.xml'),
+                                (u'Entertainment',u'http://feed43.com/4578504030588024.xml'),
-         (u'Gaming',u'http://feed43.com/8883764600355347.xml'),
+
-                                (u'Women',u'http://feed43.com/2648221746514241.xml'),
-                                (u'Instant Improver', u'http://feed43.com/1236541026275417.xml'),
 
-    #(u'Articles', u'http://feed43.com/3428534448355545.xml')
     ]
+
+

recipes/stamgasten.recipe

@@ -14,5 +14,6 @@ class AdvancedUserRecipe1347706704(BasicNewsRecipe):
     remove_empty_feeds = True
     remove_tags_before = dict(id='title')
     remove_tags_after = dict(attrs={'class':'entry-content rich-content'})
+    extra_css = 'img{border:0;padding:0;margin:0;width:100%}'
 
     feeds          = [(u'Stamgasten', u'http://toonvandriel.nl/feed/')]

recipes/television_without_pity.recipe

@@ -0,0 +1,21 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class HindustanTimes(BasicNewsRecipe):
+    title          = u'Television Without Pity'
+    language       = 'en'
+    __author__     = 'Krittika Goyal'
+    oldest_article = 1 #days
+    max_articles_per_feed = 25
+    #encoding = 'cp1252'
+    use_embedded_content = False
+
+    no_stylesheets = True
+    auto_cleanup = True
+    #auto_cleanup_keep = '//div[@class="float_right"]'
+
+
+    feeds          = [
+('News',
+ 'http://www.televisionwithoutpity.com/rss.xml'),
+]
+

recipes/the_new_age_za.recipe

@@ -0,0 +1,50 @@
+__license__   = 'GPL v3'
+__copyright__ = '2012, Darko Miletic <darko.miletic at gmail.com>'
+'''
+www.thenewage.co.za
+'''
+
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class TheNewAge_za(BasicNewsRecipe):
+    title                 = 'The New Age'
+    __author__            = 'Darko Miletic'
+    description           = "The New Age newspaper is a national daily newspaper, owned and operated by TNA Media (Pty) Ltd. TNA Media was established in June 2010 and the first publication of The New Age was on 6 December 2010. The New Age covers news from all nine provinces, along with national events, Op-Ed columns, politics, Africa and International news, sports, business, entertainment, lifestyle, science and technology."
+    publisher             = 'TNA Media (Pty.) Ltd.'
+    category              = 'news, politics, South Africa'
+    oldest_article        = 2
+    max_articles_per_feed = 200
+    no_stylesheets        = True
+    encoding              = 'utf8'
+    use_embedded_content  = False
+    auto_cleanup          = False    
+    language              = 'en_ZA'
+    remove_empty_feeds    = True
+    publication_type      = 'newspaper'
+    masthead_url          = 'http://www.thenewage.co.za/image/tnalogo.png'
+    extra_css             = """ 
+                                body{font-family: Arial,Verdana,sans-serif }
+                                img{display: block}
+                                .storyheadline{font-size: x-large; font-weight: bold}
+                            """
+
+    conversion_options = {
+                          'comment'   : description
+                        , 'tags'      : category
+                        , 'publisher' : publisher
+                        , 'language'  : language
+                        }
+
+    remove_tags = [dict(name=['object','embed','iframe','table','meta','link'])]
+    keep_only_tags = [dict(name='div', attrs={'id':['dv_headline', 'dv_story_dtls']})]
+
+
+    feeds = [
+              (u'National' , u'http://www.thenewage.co.za/rss.aspx?cat_id=1007')
+             ,(u'Provinces', u'http://www.thenewage.co.za/rss.aspx?cat_id=1008')
+             ,(u'Business' , u'http://www.thenewage.co.za/rss.aspx?cat_id=9'   )
+             ,(u'Sport'    , u'http://www.thenewage.co.za/rss.aspx?cat_id=10'  )
+             ,(u'World'    , u'http://www.thenewage.co.za/rss.aspx?cat_id=1020')
+             ,(u'Africa'   , u'http://www.thenewage.co.za/rss.aspx?cat_id=1019')
+             ,(u'Science&Tech', u'http://www.thenewage.co.za/rss.aspx?cat_id=1021')
+            ]

recipes/the_sun.recipe

@@ -8,28 +8,23 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     title          = u'The Sun UK'
     description = 'Articles from The Sun tabloid UK'
     __author__ = 'Dave Asbury'
-    # last updated 25/7/12
+    # last updated 12/10/12 added starsons remove article code
     language = 'en_GB'
     oldest_article = 1
-    max_articles_per_feed = 12
+    max_articles_per_feed = 15
     remove_empty_feeds = True
-    no_stylesheets = True
-
 
     masthead_url = 'http://www.thesun.co.uk/sol/img/global/Sun-logo.gif'
     encoding = 'UTF-8'
     remove_javascript     = True
     no_stylesheets = True
 
-
+    ignore_duplicate_articles = {'title'}
-
-    #preprocess_regexps = [
-     #   (re.compile(r'<div class="foot-copyright".*?</div>', re.IGNORECASE | re.DOTALL), lambda match: '')]
 
 
     extra_css  = '''
     body{ text-align: justify; font-family:Arial,Helvetica,sans-serif; font-size:11px; font-size-adjust:none; font-stretch:normal; font-style:normal; font-variant:normal; font-weight:normal;}
-                '''
+           '''
     keep_only_tags = [
         dict(name='div',attrs={'class' : 'intro'}),
                                 dict(name='h3'),
@@ -52,6 +47,17 @@ class AdvancedUserRecipe1325006965(BasicNewsRecipe):
     (u'Showbiz', u'http://www.thesun.co.uk/sol/homepage/showbiz/rss'),
     (u'Woman', u'http://www.thesun.co.uk/sol/homepage/woman/rss'),
     ]
+# starsons code
+    def parse_feeds (self):
+      feeds = BasicNewsRecipe.parse_feeds(self)
+      for feed in feeds:
+        for article in feed.articles[:]:
+          print 'article.title is: ', article.title
+          if 'Try out The Sun' in article.title.upper() or 'Try-out-The-Suns' in article.url:
+            feed.articles.remove(article)
+          if 'Web porn harms kids' in article.title.upper() or 'Sun-says-Web-porn' in article.url:
+            feed.articles.remove(article)
+      return feeds
 
     def get_cover_url(self):
         soup = self.index_to_soup('http://www.politicshome.com/uk/latest_frontpage.html')

recipes/the_verge.recipe

@@ -0,0 +1,21 @@
+from calibre.web.feeds.news import BasicNewsRecipe
+
+class HindustanTimes(BasicNewsRecipe):
+    title          = u'The Verge'
+    language       = 'en'
+    __author__     = 'Krittika Goyal'
+    oldest_article = 1 #days
+    max_articles_per_feed = 25
+    #encoding = 'cp1252'
+    use_embedded_content = False
+
+    no_stylesheets = True
+    auto_cleanup = True
+    auto_cleanup_keep = '//div[@class="story-image shadowbox entry-content-asset"]'
+
+
+    feeds          = [
+('News',
+ 'http://www.theverge.com/rss/index.xml'),
+]
+

setup/iso_639/it.po

@@ -12,14 +12,14 @@ msgstr ""
 "Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
 "devel@lists.alioth.debian.org>\n"
 "POT-Creation-Date: 2011-11-25 14:01+0000\n"
-"PO-Revision-Date: 2011-09-27 17:49+0000\n"
+"PO-Revision-Date: 2012-10-01 12:40+0000\n"
-"Last-Translator: Milo Casagrande <milo@casagrande.name>\n"
+"Last-Translator: Wonderfulheart <Unknown>\n"
 "Language-Team: Italian <tp@lists.linux.it>\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"X-Launchpad-Export-Date: 2011-11-26 05:21+0000\n"
+"X-Launchpad-Export-Date: 2012-10-02 05:19+0000\n"
-"X-Generator: Launchpad (build 14381)\n"
+"X-Generator: Launchpad (build 16061)\n"
 "Language: it\n"
 
 #. name for aaa
@@ -17957,7 +17957,7 @@ msgstr "Ndoola"
 
 #. name for nds
 msgid "German; Low"
-msgstr ""
+msgstr "Tedesco; Volgare"
 
 #. name for ndt
 msgid "Ndunga"

src/calibre/constants.py

@@ -4,7 +4,7 @@ __license__   = 'GPL v3'
 __copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
 __docformat__ = 'restructuredtext en'
 __appname__   = u'calibre'
-numeric_version = (0, 9, 0)
+numeric_version = (0, 9, 2)
 __version__   = u'.'.join(map(unicode, numeric_version))
 __author__    = u"Kovid Goyal <kovid@kovidgoyal.net>"
 

src/calibre/devices/android/driver.py

@@ -40,6 +40,7 @@ class ANDROID(USBMS):
                        0xca4  : HTC_BCDS,
                        0xca9  : HTC_BCDS,
                        0xcac  : HTC_BCDS,
+                       0xcba  : HTC_BCDS,
                        0xccf  : HTC_BCDS,
                        0xcd6  : HTC_BCDS,
                        0xce5  : HTC_BCDS,
@@ -54,6 +55,7 @@ class ANDROID(USBMS):
 
             # Motorola
             0x22b8 : { 0x41d9 : [0x216], 0x2d61 : [0x100], 0x2d67 : [0x100],
+                       0x2de8 : [0x229],
                        0x41db : [0x216], 0x4285 : [0x216], 0x42a3 : [0x216],
                        0x4286 : [0x216], 0x42b3 : [0x216], 0x42b4 : [0x216],
                        0x7086 : [0x0226], 0x70a8: [0x9999], 0x42c4 : [0x216],

src/calibre/devices/interface.py

@@ -634,7 +634,7 @@ class DevicePlugin(Plugin):
         of prefs['something']. Your
         method should call device_prefs.set_overrides(pref=val, pref=val, ...).
         Currently used for:
-            metadata management (prefs['manage_device_metadata'])
+        metadata management (prefs['manage_device_metadata'])
         '''
         device_prefs.set_overrides()
 

src/calibre/devices/kobo/driver.py

@@ -12,19 +12,17 @@ Originally developed by Timothy Legge <timlegge@gmail.com>.
 Extended to support Touch firmware 2.0.0 and later and newer devices by David Forrester <davidfor@internode.on.net>
 '''
 
-import os, time, calendar
+import os, time
 from contextlib import closing
 from calibre.devices.usbms.books import BookList
 from calibre.devices.usbms.books import CollectionsBookList
 from calibre.devices.kobo.books import KTCollectionsBookList
 from calibre.devices.kobo.books import Book
 from calibre.devices.kobo.books import ImageWrapper
-from calibre.devices.kobo.bookmark import Bookmark
 from calibre.devices.mime import mime_type_ext
 from calibre.devices.usbms.driver import USBMS, debug_print
 from calibre import prints
 from calibre.ptempfile import PersistentTemporaryFile
-
 from calibre.constants import DEBUG
 from calibre.utils.config import prefs
 
@@ -35,11 +33,11 @@ class KOBO(USBMS):
     gui_name = 'Kobo Reader'
     description = _('Communicate with the Kobo Reader')
     author = 'Timothy Legge and David Forrester'
-    version = (2, 0, 0)
+    version = (2, 0, 1)
 
     dbversion = 0
     fwversion = 0
-    supported_dbversion = 33
+    supported_dbversion = 62
     has_kepubs = False
 
     supported_platforms = ['windows', 'osx', 'linux']
@@ -61,7 +59,8 @@ class KOBO(USBMS):
     SUPPORTS_SUB_DIRS = True
     SUPPORTS_ANNOTATIONS = True
 
-    VIRTUAL_BOOK_EXTENSIONS = frozenset(['kobo'])
+    # "kepubs" do not have an extension. The name looks like a GUID. Using an empty string seems to work. 
+    VIRTUAL_BOOK_EXTENSIONS = frozenset(['kobo', ''])
 
     EXTRA_CUSTOMIZATION_MESSAGE = [
             _('The Kobo supports several collections including ')+\
@@ -994,6 +993,7 @@ class KOBO(USBMS):
         return USBMS.create_annotations_path(self, mdata)
 
     def get_annotations(self, path_map):
+        from calibre.devices.kobo.bookmark import Bookmark
         EPUB_FORMATS = [u'epub']
         epub_formats = set(EPUB_FORMATS)
 
@@ -1045,6 +1045,7 @@ class KOBO(USBMS):
             extension =  os.path.splitext(path_map[id])[1]
             ContentType = self.get_content_type_from_extension(extension) if extension != '' else self.get_content_type_from_path(path_map[id])
             ContentID = self.contentid_from_path(path_map[id], ContentType)
+            debug_print("get_annotations - ContentID: ",  ContentID, "ContentType: ", ContentType)
 
             bookmark_ext = extension
 
@@ -1056,6 +1057,7 @@ class KOBO(USBMS):
         return bookmarked_books
 
     def generate_annotation_html(self, bookmark):
+        import calendar
         from calibre.ebooks.BeautifulSoup import BeautifulSoup, Tag, NavigableString
         # Returns <div class="user_annotations"> ... </div>
         #last_read_location = bookmark.last_read_location
@@ -1066,7 +1068,10 @@ class KOBO(USBMS):
             try:
                 last_read = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(calendar.timegm(time.strptime(bookmark.last_read, "%Y-%m-%dT%H:%M:%S"))))
             except:
-                last_read = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(calendar.timegm(time.strptime(bookmark.last_read, "%Y-%m-%dT%H:%M:%S.%f"))))
+                try:
+                    last_read = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(calendar.timegm(time.strptime(bookmark.last_read, "%Y-%m-%dT%H:%M:%S.%f"))))
+                except:
+                    last_read = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(calendar.timegm(time.strptime(bookmark.last_read, "%Y-%m-%dT%H:%M:%SZ"))))
         else:
             #self.datetime = time.gmtime()
             last_read = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime())
@@ -1157,6 +1162,7 @@ class KOBO(USBMS):
 
         if bm.type == 'kobo_bookmark':
             mi = db.get_metadata(db_id, index_is_id=True)
+            debug_print("KOBO:add_annotation_to_library - Title: ",  mi.title)
             user_notes_soup = self.generate_annotation_html(bm.value)
             if mi.comments:
                 a_offset = mi.comments.find('<div class="user_annotations">')
@@ -1285,7 +1291,6 @@ class KOBOTOUCH(KOBO):
                           }
 
     def initialize(self):
-        debug_print("KoboTouch:initialize")
         super(KOBOTOUCH, self).initialize()
         self.bookshelvelist = []
 
@@ -1751,6 +1756,9 @@ class KOBOTOUCH(KOBO):
                 ContentID = os.path.splitext(path)[0]
                 # Remove the prefix on the file.  it could be either
                 ContentID = ContentID.replace(self._main_prefix, '')
+            elif extension == '':
+                ContentID = path
+                ContentID = ContentID.replace(self._main_prefix + self.normalize_path('.kobo/kepub/'), '')
             else:
                 ContentID = path
                 ContentID = ContentID.replace(self._main_prefix, "file:///mnt/onboard/")

src/calibre/devices/mtp/driver.py

@@ -108,10 +108,12 @@ class MTP_DEVICE(BASE):
         f = storage.find_path((self.DRIVEINFO,))
         dinfo = {}
         if f is not None:
-            stream = self.get_mtp_file(f)
             try:
+                stream = self.get_mtp_file(f)
                 dinfo = json.load(stream, object_hook=from_json)
             except:
+                prints('Failed to load existing driveinfo.calibre file, with error:')
+                traceback.print_exc()
                 dinfo = None
         if dinfo.get('device_store_uuid', None) is None:
             dinfo['device_store_uuid'] = unicode(uuid.uuid4())

src/calibre/devices/smart_device_app/driver.py

@@ -35,7 +35,7 @@ from calibre.library.server import server_config as content_server_config
 from calibre.ptempfile import PersistentTemporaryFile
 from calibre.utils.ipc import eintr_retry_call
 from calibre.utils.config import from_json, tweaks
-from calibre.utils.date import isoformat, now
+from calibre.utils.date import isoformat, now, UNDEFINED_DATE
 from calibre.utils.filenames import ascii_filename as sanitize, shorten_components_to
 from calibre.utils.mdns import (publish as publish_zeroconf, unpublish as
         unpublish_zeroconf, get_all_ips)
@@ -657,9 +657,16 @@ class SMART_DEVICE_APP(DeviceConfig, DevicePlugin):
     def _metadata_already_on_device(self, book):
         v = self.known_metadata.get(book.lpath, None)
         if v is not None:
-            return (v.get('uuid', None) == book.get('uuid', None) and
+            # Metadata is the same if the uuids match, if the last_modified dates
-                    v.get('last_modified', None) == book.get('last_modified', None) and
+            # match, and if the height of the thumbnails is the same. The last
-                    v.get('thumbnail', None) == book.get('thumbnail', None))
+            # is there to allow a device to demand a different thumbnail size
+            if (v.get('uuid', None) == book.get('uuid', None) and
+                    v.get('last_modified', None) == book.get('last_modified', None)):
+                v_thumb = v.get('thumbnail', None)
+                b_thumb = book.get('thumbnail', None)
+                if bool(v_thumb) != bool(b_thumb):
+                    return False
+                return not v_thumb or v_thumb[1] == b_thumb[1]
         return False
 
     def _set_known_metadata(self, book, remove=False):
@@ -820,6 +827,8 @@ class SMART_DEVICE_APP(DeviceConfig, DevicePlugin):
             self._debug('Device can stream metadata', self.client_can_stream_metadata)
             self.client_can_receive_book_binary = result.get('canReceiveBookBinary', False)
             self._debug('Device can receive book binary', self.client_can_stream_metadata)
+            self.client_can_delete_multiple = result.get('canDeleteMultipleBooks', False)
+            self._debug('Device can delete multiple books', self.client_can_delete_multiple)
 
             self.client_device_kind = result.get('deviceKind', '')
             self._debug('Client device kind', self.client_device_kind)
@@ -976,6 +985,14 @@ class SMART_DEVICE_APP(DeviceConfig, DevicePlugin):
                     if '_series_sort_' in result:
                         del result['_series_sort_']
                     book = self.json_codec.raw_to_book(result, SDBook, self.PREFIX)
+
+                    # If the thumbnail is the wrong size, zero the last mod date
+                    # so the metadata will be resent
+                    thumbnail = book.get('thumbnail', None)
+                    if thumbnail and not (thumbnail[0] == self.THUMBNAIL_HEIGHT or
+                                          thumbnail[1] == self.THUMBNAIL_HEIGHT):
+                        book.set('last_modified', UNDEFINED_DATE)
+
                     bl.add_book(book, replace_metadata=True)
                     if '_new_book_' in result:
                         book.set('_new_book_', True)
@@ -1109,14 +1126,24 @@ class SMART_DEVICE_APP(DeviceConfig, DevicePlugin):
         else:
             self._debug()
 
-        for path in paths:
+        if self.client_can_delete_multiple:
-            # the path has the prefix on it (I think)
+            new_paths = []
-            path = self._strip_prefix(path)
+            for path in paths:
-            opcode, result = self._call_client('DELETE_BOOK', {'lpath': path})
+                new_paths.append(self._strip_prefix(path))
-            if opcode == 'OK':
+            opcode, result = self._call_client('DELETE_BOOK', {'lpaths': new_paths})
+            for i in range(0, len(new_paths)):
+                opcode, result = self._receive_from_client(False)
                 self._debug('removed book with UUID', result['uuid'])
-            else:
+            self._debug('removed', len(new_paths), 'books')
-                raise ControlError(desc='Protocol error - delete books')
+        else:
+            for path in paths:
+                # the path has the prefix on it (I think)
+                path = self._strip_prefix(path)
+                opcode, result = self._call_client('DELETE_BOOK', {'lpath': path})
+                if opcode == 'OK':
+                    self._debug('removed book with UUID', result['uuid'])
+                else:
+                    raise ControlError(desc='Protocol error - delete books')
 
     @synchronous('sync_lock')
     def remove_books_from_metadata(self, paths, booklists):

src/calibre/ebooks/metadata/sources/base.py

@@ -26,6 +26,7 @@ msprefs.defaults['wait_after_first_identify_result'] = 30 # seconds
 msprefs.defaults['wait_after_first_cover_result'] = 60 # seconds
 msprefs.defaults['swap_author_names'] = False
 msprefs.defaults['fewer_tags'] = True
+msprefs.defaults['find_first_edition_date'] = False
 
 # Google covers are often poor quality (scans/errors) but they have high
 # resolution, so they trump covers from better sources. So make sure they

src/calibre/ebooks/metadata/sources/identify.py

@@ -120,6 +120,8 @@ class ISBNMerge(object):
                             self.log.debug(xw.tb)
                         else:
                             isbns, min_year = xw.isbns, xw.min_year
+                            if not msprefs['find_first_edition_date']:
+                                min_year = None
                 if not isbns:
                     isbns = frozenset([isbn])
                 if isbns in self.pools:

src/calibre/ebooks/mobi/writer8/main.py

@@ -106,7 +106,7 @@ class KF8Writer(object):
         not used for fonts. '''
 
         def pointer(item, oref):
-            ref = item.abshref(oref)
+            ref = urlnormalize(item.abshref(oref))
             idx = self.resources.item_map.get(ref, None)
             if idx is not None:
                 is_image = self.resources.records[idx-1][:4] not in {b'FONT'}

src/calibre/ebooks/odt/input.py

@@ -6,15 +6,19 @@ __docformat__ = 'restructuredtext en'
 '''
 Convert an ODT file into a Open Ebook
 '''
-import os
+import os, logging
 
 from lxml import etree
+from cssutils import CSSParser
+from cssutils.css import CSSRule
+
 from odf.odf2xhtml import ODF2XHTML
 from odf.opendocument import load as odLoad
 from odf.draw import Frame as odFrame, Image as odImage
 from odf.namespaces import TEXTNS as odTEXTNS
 
 from calibre import CurrentDir, walk
+from calibre.ebooks.oeb.base import _css_logger
 
 class Extract(ODF2XHTML):
 
@@ -29,14 +33,14 @@ class Extract(ODF2XHTML):
 
     def fix_markup(self, html, log):
         root = etree.fromstring(html)
-        self.epubify_markup(root, log)
         self.filter_css(root, log)
-        self.extract_css(root)
+        self.extract_css(root, log)
+        self.epubify_markup(root, log)
         html = etree.tostring(root, encoding='utf-8',
                 xml_declaration=True)
         return html
 
-    def extract_css(self, root):
+    def extract_css(self, root, log):
         ans = []
         for s in root.xpath('//*[local-name() = "style" and @type="text/css"]'):
             ans.append(s.text)
@@ -51,9 +55,21 @@ class Extract(ODF2XHTML):
             etree.SubElement(head, ns+'link', {'type':'text/css',
                 'rel':'stylesheet', 'href':'odfpy.css'})
 
-        with open('odfpy.css', 'wb') as f:
+        css = u'\n\n'.join(ans)
-            f.write((u'\n\n'.join(ans)).encode('utf-8'))
+        parser = CSSParser(loglevel=logging.WARNING,
+                            log=_css_logger)
+        self.css = parser.parseString(css, validate=False)
 
+        with open('odfpy.css', 'wb') as f:
+            f.write(css.encode('utf-8'))
+
+    def get_css_for_class(self, cls):
+        if not cls: return None
+        for rule in self.css.cssRules.rulesOfType(CSSRule.STYLE_RULE):
+            for sel in rule.selectorList:
+                q = sel.selectorText
+                if q == '.' + cls:
+                    return rule
 
     def epubify_markup(self, root, log):
         from calibre.ebooks.oeb.base import XPath, XHTML
@@ -84,16 +100,54 @@ class Extract(ODF2XHTML):
                 div.attrib['style'] = style
                 img.attrib['style'] = 'max-width: 100%; max-height: 100%'
 
-        # A div/div/img construct causes text-align:center to not work in ADE
+        # Handle anchored images. The default markup + CSS produced by
-        # so set the display of the second div to inline. This should have no
+        # odf2xhtml works with WebKit but not with ADE. So we convert the
-        # effect (apart from minor vspace issues) in a compliant HTML renderer
+        # common cases of left/right/center aligned block images to work on
-        # but it fixes the centering of the image via a text-align:center on
+        # both webkit and ADE. We detect the case of setting the side margins
-        # the first div in ADE
+        # to auto and map it to an appropriate text-align directive, which
+        # works in both WebKit and ADE.
+        # https://bugs.launchpad.net/bugs/1063207
+        # https://bugs.launchpad.net/calibre/+bug/859343
         imgpath = XPath('descendant::h:div/h:div/h:img')
         for img in imgpath(root):
             div2 = img.getparent()
             div1 = div2.getparent()
-            if len(div1) == len(div2) == 1:
+            if (len(div1), len(div2)) != (1, 1): continue
+            cls = div1.get('class', '')
+            first_rules = filter(None, [self.get_css_for_class(x) for x in
+                cls.split()])
+            has_align = False
+            for r in first_rules:
+                if r.style.getProperty(u'text-align') is not None:
+                    has_align = True
+            ml = mr = None
+            if not has_align:
+                aval = None
+                cls = div2.get(u'class', u'')
+                rules = filter(None, [self.get_css_for_class(x) for x in
+                    cls.split()])
+                for r in rules:
+                    ml = r.style.getPropertyCSSValue(u'margin-left') or ml
+                    mr = r.style.getPropertyCSSValue(u'margin-right') or mr
+                    ml = getattr(ml, 'value', None)
+                    mr = getattr(mr, 'value', None)
+                if ml == mr == u'auto':
+                    aval = u'center'
+                elif ml == u'auto' and mr != u'auto':
+                    aval = 'right'
+                elif ml != u'auto' and mr == u'auto':
+                    aval = 'left'
+                if aval is not None:
+                    style = div1.attrib.get('style', '').strip()
+                    if style and not style.endswith(';'):
+                        style = style + ';'
+                    style += 'text-align:%s'%aval
+                    has_align = True
+                    div1.attrib['style'] = style
+
+            if has_align:
+                # This is needed for ADE, without it the text-align has no
+                # effect
                 style = div2.attrib['style']
                 div2.attrib['style'] = 'display:inline;'+style
 

src/calibre/gui2/actions/edit_metadata.py

@@ -459,9 +459,10 @@ class EditMetadataAction(InterfaceAction):
                 if src_value:
                   src_index = db.get_custom_extra(src_id, num=colnum, index_is_id=True)
                   db.set_custom(dest_id, src_value, num=colnum, extra=src_index)
-              if db.field_metadata[key]['datatype'] == 'text' \
+              if (db.field_metadata[key]['datatype'] == 'enumeration' or
-                and not db.field_metadata[key]['is_multiple'] \
+                        (db.field_metadata[key]['datatype'] == 'text' and
-                and not dest_value:
+                         not db.field_metadata[key]['is_multiple'])
+                    and not dest_value):
                 db.set_custom(dest_id, src_value, num=colnum)
               if db.field_metadata[key]['datatype'] == 'text' \
                 and db.field_metadata[key]['is_multiple']:

src/calibre/gui2/add.py

@@ -11,7 +11,7 @@ from calibre.gui2.dialogs.progress import ProgressDialog
 from calibre.gui2 import (question_dialog, error_dialog, info_dialog, gprefs,
         warning_dialog, available_width)
 from calibre.ebooks.metadata.opf2 import OPF
-from calibre.ebooks.metadata import MetaInformation
+from calibre.ebooks.metadata import MetaInformation, authors_to_string
 from calibre.constants import preferred_encoding, filesystem_encoding, DEBUG
 from calibre.utils.config import prefs
 from calibre import prints, force_unicode, as_unicode
@@ -382,12 +382,25 @@ class Adder(QObject): # {{{
         if not duplicates:
             return self.duplicates_processed()
         self.pd.hide()
-        files = [_('%(title)s by %(author)s')%dict(title=x[0].title,
+        duplicate_message = []
-            author=x[0].format_field('authors')[1]) for x in duplicates]
+        for x in duplicates:
+            duplicate_message.append(_('Already in calibre:'))
+            matching_books = self.db.books_with_same_title(x[0])
+            for book_id in matching_books:
+                aut = [a.replace('|', ',') for a in (self.db.authors(book_id,
+                    index_is_id=True) or '').split(',')]
+                duplicate_message.append('\t'+ _('%(title)s by %(author)s')%
+                        dict(title=self.db.title(book_id, index_is_id=True),
+                        author=authors_to_string(aut)))
+            duplicate_message.append(_('You are trying to add:'))
+            duplicate_message.append('\t'+_('%(title)s by %(author)s')%
+                    dict(title=x[0].title,
+                    author=x[0].format_field('authors')[1]))
+            duplicate_message.append('')
         if question_dialog(self._parent, _('Duplicates found!'),
                         _('Books with the same title as the following already '
-                        'exist in the database. Add them anyway?'),
+                        'exist in calibre. Add them anyway?'),
-                        '\n'.join(files)):
+                        '\n'.join(duplicate_message)):
             pd = QProgressDialog(_('Adding duplicates...'), '', 0, len(duplicates),
                     self._parent)
             pd.setCancelButton(None)

src/calibre/gui2/convert/pdf_output.ui

@@ -14,7 +14,7 @@
    <string>Form</string>
   </property>
   <layout class="QFormLayout" name="formLayout">
-   <item row="0" column="0">
+   <item row="1" column="0">
     <widget class="QLabel" name="label">
      <property name="text">
       <string>&amp;Paper Size:</string>
@@ -24,10 +24,10 @@
      </property>
     </widget>
    </item>
-   <item row="0" column="1">
+   <item row="1" column="1">
     <widget class="QComboBox" name="opt_paper_size"/>
    </item>
-   <item row="1" column="0">
+   <item row="2" column="0">
     <widget class="QLabel" name="label_2">
      <property name="text">
       <string>&amp;Orientation:</string>
@@ -37,10 +37,10 @@
      </property>
     </widget>
    </item>
-   <item row="1" column="1">
+   <item row="2" column="1">
     <widget class="QComboBox" name="opt_orientation"/>
    </item>
-   <item row="2" column="0">
+   <item row="3" column="0">
     <widget class="QLabel" name="label_3">
      <property name="text">
       <string>&amp;Custom size:</string>
@@ -50,17 +50,17 @@
      </property>
     </widget>
    </item>
-   <item row="2" column="1">
+   <item row="3" column="1">
     <widget class="QLineEdit" name="opt_custom_size"/>
    </item>
-   <item row="3" column="0" colspan="2">
+   <item row="4" column="0" colspan="2">
     <widget class="QCheckBox" name="opt_preserve_cover_aspect_ratio">
      <property name="text">
       <string>Preserve &amp;aspect ratio of cover</string>
      </property>
     </widget>
    </item>
-   <item row="10" column="0">
+   <item row="11" column="0">
     <spacer name="verticalSpacer">
      <property name="orientation">
       <enum>Qt::Vertical</enum>
@@ -73,7 +73,7 @@
      </property>
     </spacer>
    </item>
-   <item row="4" column="0">
+   <item row="5" column="0">
     <widget class="QLabel" name="label_4">
      <property name="text">
       <string>Se&amp;rif family:</string>
@@ -83,10 +83,10 @@
      </property>
     </widget>
    </item>
-   <item row="4" column="1">
+   <item row="5" column="1">
     <widget class="QFontComboBox" name="opt_pdf_serif_family"/>
    </item>
-   <item row="5" column="0">
+   <item row="6" column="0">
     <widget class="QLabel" name="label_5">
      <property name="text">
       <string>&amp;Sans family:</string>
@@ -96,10 +96,10 @@
      </property>
     </widget>
    </item>
-   <item row="5" column="1">
+   <item row="6" column="1">
     <widget class="QFontComboBox" name="opt_pdf_sans_family"/>
    </item>
-   <item row="6" column="0">
+   <item row="7" column="0">
     <widget class="QLabel" name="label_6">
      <property name="text">
       <string>&amp;Monospace family:</string>
@@ -109,10 +109,10 @@
      </property>
     </widget>
    </item>
-   <item row="6" column="1">
+   <item row="7" column="1">
     <widget class="QFontComboBox" name="opt_pdf_mono_family"/>
    </item>
-   <item row="7" column="0">
+   <item row="8" column="0">
     <widget class="QLabel" name="label_7">
      <property name="text">
       <string>S&amp;tandard font:</string>
@@ -122,10 +122,10 @@
      </property>
     </widget>
    </item>
-   <item row="7" column="1">
+   <item row="8" column="1">
     <widget class="QComboBox" name="opt_pdf_standard_font"/>
    </item>
-   <item row="8" column="0">
+   <item row="9" column="0">
     <widget class="QLabel" name="label_8">
      <property name="text">
       <string>Default font si&amp;ze:</string>
@@ -135,14 +135,14 @@
      </property>
     </widget>
    </item>
-   <item row="8" column="1">
+   <item row="9" column="1">
     <widget class="QSpinBox" name="opt_pdf_default_font_size">
      <property name="suffix">
       <string> px</string>
      </property>
     </widget>
    </item>
-   <item row="9" column="0">
+   <item row="10" column="0">
     <widget class="QLabel" name="label_9">
      <property name="text">
       <string>Monospace &amp;font size:</string>
@@ -152,13 +152,23 @@
      </property>
     </widget>
    </item>
-   <item row="9" column="1">
+   <item row="10" column="1">
     <widget class="QSpinBox" name="opt_pdf_mono_font_size">
      <property name="suffix">
       <string> px</string>
      </property>
     </widget>
    </item>
+   <item row="0" column="0" colspan="2">
+    <widget class="QLabel" name="label_10">
+     <property name="text">
+      <string>&lt;b&gt;Note:&lt;/b&gt; The paper size settings below only take effect if you have set the output profile to the default output profile. Otherwise the output profile will override these settings.</string>
+     </property>
+     <property name="wordWrap">
+      <bool>true</bool>
+     </property>
+    </widget>
+   </item>
   </layout>
  </widget>
  <resources/>

src/calibre/gui2/device.py

@@ -1695,7 +1695,9 @@ class DeviceMixin(object): # {{{
                 book.in_library = None
                 if getattr(book, 'uuid', None) in self.db_book_uuid_cache:
                     id_ = db_book_uuid_cache[book.uuid]
-                    if update_metadata:
+                    if (update_metadata and
+                            db.metadata_last_modified(id_, index_is_id=True) !=
+                                        getattr(book, 'last_modified', None)):
                         mi = db.get_metadata(id_, index_is_id=True,
                                              get_cover=get_covers)
                         book.smart_update(mi, replace_metadata=True)

src/calibre/gui2/device_drivers/mtp_config.py

@@ -146,7 +146,7 @@ class SendToConfig(QWidget): # {{{
     def browse(self):
         b = Browser(self.device.filesystem_cache, show_files=False,
                 parent=self)
-        if b.exec_() == b.Accepted:
+        if b.exec_() == b.Accepted and b.current_item is not None:
             sid, path = b.current_item
             self.t.setText('/'.join(path[1:]))
 
@@ -250,7 +250,7 @@ class Rule(QWidget):
     def browse(self):
         b = Browser(self.device.filesystem_cache, show_files=False,
                 parent=self)
-        if b.exec_() == b.Accepted:
+        if b.exec_() == b.Accepted and b.current_item is not None:
             sid, path = b.current_item
             self.folder.setText('/'.join(path[1:]))
 

src/calibre/gui2/library/views.py

@@ -511,6 +511,7 @@ class BooksView(QTableView): # {{{
             except:
                 # Ignore invalid tweak values as users seem to often get them
                 # wrong
+                print('Ignoring invalid sort_columns_at_startup tweak, with error:')
                 import traceback
                 traceback.print_exc()
             old_state['sort_history'] = sh

src/calibre/gui2/metadata/single_download.py

@@ -678,11 +678,12 @@ class CoversModel(QAbstractListModel): # {{{
         good = []
         pmap = {}
         dcovers = sorted(self.covers[1:], key=self.cover_keygen, reverse=True)
+        cmap = {x:self.covers.index(x) for x in self.covers}
         for i, x in enumerate(self.covers[0:1] + dcovers):
             if not x[-1]:
                 good.append(x)
                 if i > 0:
-                    plugin = self.plugin_for_index(i)
+                    plugin = self.plugin_for_index(cmap[x])
                     pmap[plugin] = len(good) - 1
         self.covers = good
         self.plugin_map = pmap

src/calibre/gui2/preferences/metadata_sources.py

@@ -296,6 +296,7 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
         r('wait_after_first_cover_result', msprefs)
         r('swap_author_names', msprefs)
         r('fewer_tags', msprefs)
+        r('find_first_edition_date', msprefs)
 
         self.configure_plugin_button.clicked.connect(self.configure_plugin)
         self.sources_model = SourcesModel(self)

src/calibre/gui2/preferences/metadata_sources.ui

@@ -7,7 +7,7 @@
     <x>0</x>
     <y>0</y>
     <width>781</width>
-    <height>394</height>
+    <height>439</height>
    </rect>
   </property>
   <property name="windowTitle">
@@ -21,7 +21,7 @@
     <widget class="QStackedWidget" name="stack">
      <widget class="QWidget" name="page">
       <layout class="QGridLayout" name="gridLayout">
-       <item row="0" column="0" rowspan="7">
+       <item row="0" column="0" rowspan="8">
         <widget class="QGroupBox" name="groupBox">
          <property name="title">
           <string>Metadata sources</string>
@@ -104,22 +104,22 @@
           </item>
           <item row="2" column="0">
            <widget class="QPushButton" name="select_default_button">
+            <property name="toolTip">
+             <string>Restore your own subset of checked fields that you define using the 'Set as default' button</string>
+            </property>
             <property name="text">
              <string>&amp;Select default</string>
             </property>
-             <property name="toolTip">
-             <string>Restore your own subset of checked fields that you define using the 'Set as default' button</string>
-            </property>
            </widget>
           </item>
           <item row="2" column="1">
            <widget class="QPushButton" name="set_as_default_button">
+            <property name="toolTip">
+             <string>Store the currently checked fields as a default you can restore using the 'Select default' button</string>
+            </property>
             <property name="text">
              <string>&amp;Set as default</string>
             </property>
-             <property name="toolTip">
-             <string>Store the currently checked fields as a default you can restore using the 'Select default' button</string>
-            </property>
            </widget>
           </item>
          </layout>
@@ -139,7 +139,7 @@
          </property>
         </widget>
        </item>
-       <item row="4" column="1">
+       <item row="5" column="1">
         <widget class="QLabel" name="label_2">
          <property name="text">
           <string>Max. number of &amp;tags to download:</string>
@@ -149,10 +149,10 @@
          </property>
         </widget>
        </item>
-       <item row="4" column="2">
+       <item row="5" column="2">
         <widget class="QSpinBox" name="opt_max_tags"/>
        </item>
-       <item row="5" column="1">
+       <item row="6" column="1">
         <widget class="QLabel" name="label_3">
          <property name="text">
           <string>Max. &amp;time to wait after first match is found:</string>
@@ -162,14 +162,14 @@
          </property>
         </widget>
        </item>
-       <item row="5" column="2">
+       <item row="6" column="2">
         <widget class="QSpinBox" name="opt_wait_after_first_identify_result">
          <property name="suffix">
           <string> secs</string>
          </property>
         </widget>
        </item>
-       <item row="6" column="1">
+       <item row="7" column="1">
         <widget class="QLabel" name="label_4">
          <property name="text">
           <string>Max. time to wait after first &amp;cover is found:</string>
@@ -179,14 +179,14 @@
          </property>
         </widget>
        </item>
-       <item row="6" column="2">
+       <item row="7" column="2">
         <widget class="QSpinBox" name="opt_wait_after_first_cover_result">
          <property name="suffix">
           <string> secs</string>
          </property>
         </widget>
        </item>
-       <item row="3" column="1" colspan="2">
+       <item row="4" column="1" colspan="2">
         <widget class="QCheckBox" name="opt_fewer_tags">
          <property name="toolTip">
           <string>&lt;p&gt;Different metadata sources have different sets of tags for the same book. If this option is checked, then calibre will use the smaller tag sets. These tend to be more like genres, while the larger tag sets tend to describe the books content.
@@ -197,6 +197,13 @@
          </property>
         </widget>
        </item>
+       <item row="3" column="1" colspan="2">
+        <widget class="QCheckBox" name="opt_find_first_edition_date">
+         <property name="text">
+          <string>Use published date of &quot;first edition&quot; (from worldcat.org)</string>
+         </property>
+        </widget>
+       </item>
       </layout>
      </widget>
      <widget class="QWidget" name="page_2"/>

src/calibre/gui2/store/stores/woblink_plugin.py

@@ -6,7 +6,6 @@ __license__ = 'GPL 3'
 __copyright__ = '2011-2012, Tomasz Długosz <tomek3d@gmail.com>'
 __docformat__ = 'restructuredtext en'
 
-import copy
 import re
 import urllib
 from contextlib import closing
@@ -74,7 +73,7 @@ class WoblinkStore(BasicStoreConfig, StorePlugin):
                 s.author = author.strip()
                 s.price = price + ' zł'
                 s.detail_item = id.strip()
-                
+
                 if 'epub_drm' in formats:
                     s.drm = SearchResult.DRM_LOCKED
                     s.formats = 'EPUB'
@@ -84,7 +83,7 @@ class WoblinkStore(BasicStoreConfig, StorePlugin):
                 elif 'pdf' in formats:
                     s.drm = SearchResult.DRM_LOCKED
                     s.formats = 'PDF'
-                
+
                     counter -= 1
                     yield s
                 else:
@@ -93,6 +92,6 @@ class WoblinkStore(BasicStoreConfig, StorePlugin):
                         formats.remove('MOBI_nieb')
                         formats.append('MOBI')
                     s.formats = ', '.join(formats).upper()
-                
+
                     counter -= 1
                     yield s

src/calibre/gui2/viewer/config.py

@@ -55,6 +55,10 @@ def config(defaults=None):
                 '0 and 1.'))
     c.add_opt('fullscreen_clock', default=False, action='store_true',
             help=_('Show a clock in fullscreen mode.'))
+    c.add_opt('fullscreen_pos', default=False, action='store_true',
+            help=_('Show reading position in fullscreen mode.'))
+    c.add_opt('fullscreen_scrollbar', default=True, action='store_false',
+            help=_('Show the scrollbar in fullscreen mode.'))
     c.add_opt('cols_per_screen', default=1)
     c.add_opt('use_book_margins', default=False, action='store_true')
     c.add_opt('top_margin', default=20)
@@ -201,6 +205,8 @@ class ConfigDialog(QDialog, Ui_Dialog):
         self.hyphenate_default_lang.setEnabled(opts.hyphenate)
         self.opt_fit_images.setChecked(opts.fit_images)
         self.opt_fullscreen_clock.setChecked(opts.fullscreen_clock)
+        self.opt_fullscreen_scrollbar.setChecked(opts.fullscreen_scrollbar)
+        self.opt_fullscreen_pos.setChecked(opts.fullscreen_pos)
         self.opt_cols_per_screen.setValue(opts.cols_per_screen)
         self.opt_override_book_margins.setChecked(not opts.use_book_margins)
         for x in ('top', 'bottom', 'side'):
@@ -271,6 +277,8 @@ class ConfigDialog(QDialog, Ui_Dialog):
         c.set('line_scrolling_stops_on_pagebreaks',
                 self.opt_line_scrolling_stops_on_pagebreaks.isChecked())
         c.set('fullscreen_clock', self.opt_fullscreen_clock.isChecked())
+        c.set('fullscreen_pos', self.opt_fullscreen_pos.isChecked())
+        c.set('fullscreen_scrollbar', self.opt_fullscreen_scrollbar.isChecked())
         c.set('cols_per_screen', int(self.opt_cols_per_screen.value()))
         c.set('use_book_margins', not
                 self.opt_override_book_margins.isChecked())

src/calibre/gui2/viewer/config.ui

@@ -347,8 +347,8 @@ QToolBox::tab:hover {
            <rect>
             <x>0</x>
             <y>0</y>
-            <width>313</width>
+            <width>811</width>
-            <height>64</height>
+            <height>352</height>
            </rect>
           </property>
           <attribute name="label">
@@ -388,6 +388,20 @@ QToolBox::tab:hover {
              </property>
             </widget>
            </item>
+           <item row="3" column="0" colspan="2">
+            <widget class="QCheckBox" name="opt_fullscreen_pos">
+             <property name="text">
+              <string>Show reading &amp;position in full screen mode</string>
+             </property>
+            </widget>
+           </item>
+           <item row="2" column="0" colspan="2">
+            <widget class="QCheckBox" name="opt_fullscreen_scrollbar">
+             <property name="text">
+              <string>Show &amp;scrollbar in full screen mode</string>
+             </property>
+            </widget>
+           </item>
           </layout>
          </widget>
          <widget class="QWidget" name="page_6">

src/calibre/gui2/viewer/documentview.py

@@ -143,6 +143,8 @@ class Document(QWebPage): # {{{
         # Leave some space for the scrollbar and some border
         self.max_fs_width = min(opts.max_fs_width, screen_width-50)
         self.fullscreen_clock = opts.fullscreen_clock
+        self.fullscreen_scrollbar = opts.fullscreen_scrollbar
+        self.fullscreen_pos = opts.fullscreen_pos
         self.use_book_margins = opts.use_book_margins
         self.cols_per_screen = opts.cols_per_screen
         self.side_margin = opts.side_margin
@@ -477,7 +479,7 @@ class DocumentView(QWebView): # {{{
         d = self.document
         self.unimplemented_actions = list(map(self.pageAction,
             [d.DownloadImageToDisk, d.OpenLinkInNewWindow, d.DownloadLinkToDisk,
-                d.OpenImageInNewWindow, d.OpenLink]))
+                d.OpenImageInNewWindow, d.OpenLink, d.Reload]))
         self.dictionary_action = QAction(QIcon(I('dictionary.png')),
                 _('&Lookup in dictionary'), self)
         self.dictionary_action.setShortcut(Qt.CTRL+Qt.Key_L)

src/calibre/gui2/viewer/main.py

@@ -112,6 +112,8 @@ class Metadata(QLabel):
 
 class DoubleSpinBox(QDoubleSpinBox):
 
+    value_changed = pyqtSignal(object, object)
+
     def __init__(self, *args, **kwargs):
         QDoubleSpinBox.__init__(self, *args, **kwargs)
         self.tt = _('Position in book')
@@ -123,6 +125,7 @@ class DoubleSpinBox(QDoubleSpinBox):
         self.setToolTip(self.tt +
                 ' [{0:.0%}]'.format(float(val)/self.maximum()))
         self.blockSignals(False)
+        self.value_changed.emit(self.value(), self.maximum())
 
 class Reference(QLineEdit):
 
@@ -185,6 +188,7 @@ class EbookViewer(MainWindow, Ui_EbookViewer):
         self.pos.setDecimals(1)
         self.pos.setSuffix('/'+_('Unknown')+'     ')
         self.pos.setMinimum(1.)
+        self.pos.value_changed.connect(self.update_pos_label)
         self.splitter.setCollapsible(0, False)
         self.splitter.setCollapsible(1, False)
         self.pos.setMinimumWidth(150)
@@ -302,9 +306,9 @@ class EbookViewer(MainWindow, Ui_EbookViewer):
         self.clock_label = QLabel('99:99', self)
         self.clock_label.setVisible(False)
         self.clock_label.setFocusPolicy(Qt.NoFocus)
-        self.clock_label_style = '''
+        self.info_label_style = '''
             QLabel {
-                text-align: right;
+                text-align: center;
                 border-width: 1px;
                 border-style: solid;
                 border-radius: 8px;
@@ -314,6 +318,10 @@ class EbookViewer(MainWindow, Ui_EbookViewer):
                 font-size: larger;
                 padding: 5px;
         }'''
+        self.original_frame_style = self.frame.frameStyle()
+        self.pos_label = QLabel('2000/4000', self)
+        self.pos_label.setVisible(False)
+        self.pos_label.setFocusPolicy(Qt.NoFocus)
         self.clock_timer = QTimer(self)
         self.clock_timer.timeout.connect(self.update_clock)
         self.esc_full_screen_action = a = QAction(self)
@@ -480,11 +488,15 @@ class EbookViewer(MainWindow, Ui_EbookViewer):
         self.window_mode_changed = 'fullscreen'
         self.tool_bar.setVisible(False)
         self.tool_bar2.setVisible(False)
+        if not self.view.document.fullscreen_scrollbar:
+            self.vertical_scrollbar.setVisible(False)
+            self.frame.layout().setSpacing(0)
         self._original_frame_margins = (
             self.centralwidget.layout().contentsMargins(),
             self.frame.layout().contentsMargins())
         self.frame.layout().setContentsMargins(0, 0, 0, 0)
         self.centralwidget.layout().setContentsMargins(0, 0, 0, 0)
+        self.frame.setFrameStyle(self.frame.NoFrame|self.frame.Plain)
 
         super(EbookViewer, self).showFullScreen()
 
@@ -505,27 +517,54 @@ class EbookViewer(MainWindow, Ui_EbookViewer):
         self.view.document.switch_to_fullscreen_mode()
         if self.view.document.fullscreen_clock:
             self.show_clock()
+        if self.view.document.fullscreen_pos:
+            self.show_pos_label()
 
     def show_clock(self):
         self.clock_label.setVisible(True)
-        self.clock_label.setText('99:99 AA')
+        self.clock_label.setText(QTime(22, 33,
+            33).toString(Qt.SystemLocaleShortDate))
         self.clock_timer.start(1000)
-        self.clock_label.setStyleSheet(self.clock_label_style%(
+        self.clock_label.setStyleSheet(self.info_label_style%(
                 'rgba(0, 0, 0, 0)', self.view.document.colors()[1]))
         self.clock_label.resize(self.clock_label.sizeHint())
         sw = QApplication.desktop().screenGeometry(self.view)
-        self.clock_label.move(sw.width() - self.vertical_scrollbar.width() - 15
+        vswidth = (self.vertical_scrollbar.width() if
+                self.vertical_scrollbar.isVisible() else 0)
+        self.clock_label.move(sw.width() - vswidth - 15
                 - self.clock_label.width(), sw.height() -
                 self.clock_label.height()-10)
         self.update_clock()
 
+    def show_pos_label(self):
+        self.pos_label.setVisible(True)
+        self.pos_label.setStyleSheet(self.info_label_style%(
+                'rgba(0, 0, 0, 0)', self.view.document.colors()[1]))
+        sw = QApplication.desktop().screenGeometry(self.view)
+        self.pos_label.move(15, sw.height() - self.pos_label.height()-10)
+        self.update_pos_label()
+
     def update_clock(self):
-        self.clock_label.setText(QTime.currentTime().toString('h:mm a'))
+        self.clock_label.setText(QTime.currentTime().toString(Qt.SystemLocaleShortDate))
+
+    def update_pos_label(self, *args):
+        if self.pos_label.isVisible():
+            try:
+                value, maximum = args
+            except:
+                value, maximum = self.pos.value(), self.pos.maximum()
+            text = '%g/%g'%(value, maximum)
+            self.pos_label.setText(text)
+            self.pos_label.resize(self.pos_label.sizeHint())
 
     def showNormal(self):
         self.view.document.page_position.save()
         self.clock_label.setVisible(False)
+        self.pos_label.setVisible(False)
+        self.frame.setFrameStyle(self.original_frame_style)
+        self.frame.layout().setSpacing(-1)
         self.clock_timer.stop()
+        self.vertical_scrollbar.setVisible(True)
         self.window_mode_changed = 'normal'
         self.esc_full_screen_action.setEnabled(False)
         self.tool_bar.setVisible(True)

src/calibre/library/database2.py

@@ -3664,7 +3664,7 @@ books_series_link      feeds
                 if not ext:
                     continue
                 ext = ext[1:].lower()
-                if ext not in BOOK_EXTENSIONS:
+                if ext not in BOOK_EXTENSIONS and ext != 'opf':
                     continue
 
                 key = os.path.splitext(path)[0]

src/calibre/library/server/opds.py

@@ -594,6 +594,9 @@ class OPDSServer(object):
             meta = category_meta.get(category, None)
             if meta is None:
                 continue
+            if category_meta.is_custom_field(category) and \
+                                category not in custom_fields_to_display(self.db):
+                continue
             cats.append((meta['name'], meta['name'], 'N'+category))
         updated = self.db.last_modified()