Cutlery/calibre
1
0
Fork 0
mirror of https://github.com/kovidgoyal/calibre.git synced 2025-07-09 03:04:10 -04:00
Code Issues Packages Projects Releases Wiki Activity

0.7.54

Browse Source
This commit is contained in:
GRiker 2011-04-08 13:34:22 -06:00
commit 99af4d355c
150 changed files with 42398 additions and 35010 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

65
Changelog.yaml
View File

@ -19,18 +19,77 @@
# new recipes:
# - title:
- version: 0.7.54
date: 2011-04-08
new features:
- title: "New output format, HTMLZ which is a single HTML file with its associated images/stylesheets in a zipped up file"
description: "Useful when you want to convert your ebook into a single HTML file for easy editing. Note that this output plugin is still new and needs testing"
- title: "When dealing with ZIP/RAR archives, use the file header rather than the file extension to detrmine the file type, when possible. This fixes the common case of CBZ files being actually cbr files and vice versa"
- title: "Support for the Motorola Atrix"
- title: "Allow the icons in the toolbar to be turned off completely via Preferences->Look & Feel"
- title: "When downloading metadata use the gzip transfer encoding when possible for a speedup."
tickets: [749304]
bug fixes:
- title: "Conversion pipeline: Workaround for bug in lxml that causes a massive mem leak on windows and OS X when the input document contains non ASCII CSS selectors."
tickets: [754555]
- title: "Conversion pipeline: Handle inline <style> tags that put all the actual CSS inside an XML comment."
tickets: [750063]
- title: "The 'Choose Library' button now shows its popup menu when you already have more than one library instead of the dialog to create a new library"
tickets: [754154]
- title: "Apply all content server setting when clicking the Start Server button in Preferences->Sharing over the net"
tickets: [753122]
- title: "Fix content server breaking if its restriction is set to a saved search that was deleted"
tickets: [751950]
- title: "Fix detection of PocketBook with 2.0.6 firmware on windows"
tickets: [750336]
- title: "ODT Input: Fix handling of the <text:s> element."
tickets: [749655]
- title: "MOBI Output: Don't use self closed tags"
- title: "Fix book details popup becoming too tall if there is a lot of metadata"
- title: "Fix new PDF engine crashing on PDF files with embedded fonts with null names"
improved recipes:
- Kommersant
- Perfil
- Times of India
- IHT
- Guardian
new recipes:
- title: "Al Ahram"
authors: Hassan Williamson
- title: "F-Secure and developpez.com"
authors: louhike
- version: 0.7.53
date: 2011-04-01
new features:
- title: "Email delivery: You can now specify a subject that calibre will use when sending emails per email account, configured in Preferences->Sending by email. The subject is a template of the same kind used in Save to Disk, etc. So youcan specift the title/authors/series/whatever in the template."
- title: "Email delivery: You can now specify a subject that calibre will use when sending emails per email account, configured in Preferences->Sending by email. The subject is a template of the same kind used in Save to Disk, etc. So you can specift the title/authors/series/whatever in the template."
tickets: [743535]
- title: "Apple driver: When an iDevice is detected, inform the user about the Connect to iTunes method instead of trying to connect directly to the device, as the latter can be buggy. See http://www.mobileread.com/forums/showthread.php?t=127883 for details"
- title: "SONY driver: Search for books on the device in all directories not just database/media/books. This can be turned off by customizing the SONY plugin in Preferences->Plugins"
- title: "EPUB Output: Remove any margins specified via an Adobe page template in the input document. This means that the margins psecified in calibre are more likely to be the actual margins used."
- title: "EPUB Output: Remove any margins specified via an Adobe page template in the input document. This means that the margins specified in calibre are more likely to be the actual margins used."
- title: "When reading metadata from filenames, allow publisher and published date to be read from the filename"
tickets: [744020]
@ -49,6 +108,8 @@
- title: "FB2 Output: Option to set the FB2 genre explicitly."
tickets: [743178]
- title: "Plugin developers: calibre now has a new plugin API, see http://calibre-ebook.com/user_manual/creating_plugins.html. Your existing plugins should continue to work, but it would be good to test them to make sure."
bug fixes:
- title: "Fix text color in the search bar set to black instead of the system font color"

8
INSTALL
View File

@ -1,6 +1,9 @@
calibre supports installation from source, only on Linux.
On Windows and OS X use the provided installers and use
the facilities of the calibre-debug command to hack on the calibre source.
Note that you *do not* need to install from source to hack on
the calibre source code. To get started with calibre development,
use a normal calibre install and follow the instructions at
http://calibre-ebook.com/user_manual/develop.html
On Linux, there are two kinds of installation from source possible.
Note that both kinds require lots of dependencies as well as a
@ -45,3 +48,4 @@ This type of install can be run with the command::
sudo python setup.py develop
Use the -h flag for help on the develop command.

2
README
View File

@ -7,7 +7,7 @@ reading. It is cross platform, running on Linux, Windows and OS X.
For screenshots: https://calibre-ebook.com/demo
For installation/usage instructions please see
http://calibre-ebook.com
http://calibre-ebook.com/user_manual
For source code access:
bzr branch lp:calibre

62
recipes/al_ahram.recipe Normal file
View File

@ -0,0 +1,62 @@
# coding=utf-8
__license__ = 'GPL v3'
__copyright__ = '2011, Hassan Williamson <haz at hazrpg.co.uk>'
'''
ahram.org.eg
'''
from calibre.web.feeds.recipes import BasicNewsRecipe
class AlAhram(BasicNewsRecipe):
title = 'Al-Ahram'
__author__ = 'Hassan Williamson'
description = 'News from Egypt in Arabic.'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
#delay = 1
use_embedded_content = False
encoding = 'utf8'
publisher = 'Al-Ahram'
category = 'News'
language = 'ar'
publication_type = 'newsportal'
extra_css = ' body{ font-family: Verdana,Helvetica,Arial,sans-serif; direction: rtl; } .txtTitle{ font-weight: bold; } '
keep_only_tags = [
dict(name='div', attrs={'class':['bbcolright']})
]
remove_tags = [
dict(name='div', attrs={'class':['bbnav', 'bbsp']}),
dict(name='div', attrs={'id':['AddThisButton']})
]
remove_attributes = [
'width','height'
]
feeds = [
(u'الأولى', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=25'),
(u'مصر', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=27'),
(u'المحافظات', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=29'),
(u'الوطن العربي', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=31'),
(u'العالم', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=26'),
(u'تقارير المراسلين', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=2'),
(u'تحقيقات', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=3'),
(u'قضايا واراء', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=4'),
(u'اقتصاد', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=5'),
(u'رياضة', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=6'),
(u'حوادث', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=38'),
(u'دنيا الثقافة', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=7'),
(u'المراة والطفل', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=8'),
(u'يوم جديد', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=9'),
(u'الكتاب', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=10'),
(u'الاعمدة', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=11'),
(u'أراء حرة', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=59'),
(u'ملفات الاهرام', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=12'),
(u'بريد الاهرام', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=15'),
(u'الاخيرة', 'http://www.ahram.org.eg/RssXml.aspx?CategoryID=16'),
]

21
recipes/developpez.recipe Normal file
View File

@ -0,0 +1,21 @@
from calibre.web.feeds.news import BasicNewsRecipe
class AdvancedUserRecipe1301849956(BasicNewsRecipe):
title = u'Developpez.com'
description = u'Toutes les news du site Developpez.com'
publisher = u'Developpez.com'
timefmt = ' [%a, %d %b, %Y]'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
encoding = 'ISO-8859-1'
language = 'fr'
__author__ = 'louhike'
remove_javascript = True
keep_only_tags = [dict(name='div', attrs={'class':'content'})]
feeds = [(u'Tous les articles', u'http://www.developpez.com/index/rss')]
def get_cover_url(self):
return 'http://javascript.developpez.com/template/images/logo.gif'

3
recipes/economist.recipe
View File

@ -18,7 +18,8 @@ class Economist(BasicNewsRecipe):
__author__ = "Kovid Goyal"
INDEX = 'http://www.economist.com/printedition'
description = 'Global news and current affairs from a European perspective.'
description = ('Global news and current affairs from a European'
' perspective. Best downloaded on Friday mornings (GMT)')
oldest_article = 7.0
cover_url = 'http://www.economist.com/images/covers/currentcoverus_large.jpg'

3
recipes/economist_free.recipe
View File

@ -11,7 +11,8 @@ class Economist(BasicNewsRecipe):
language = 'en'
__author__ = "Kovid Goyal"
description = ('Global news and current affairs from a European perspective.'
description = ('Global news and current affairs from a European'
' perspective. Best downloaded on Friday mornings (GMT).'
' Much slower than the print edition based version.')
oldest_article = 7.0

22
recipes/f_secure.recipe Normal file
View File

@ -0,0 +1,22 @@
from calibre.web.feeds.news import BasicNewsRecipe
class AdvancedUserRecipe1301860159(BasicNewsRecipe):
title = u'F-Secure Weblog'
language = 'en'
__author__ = 'louhike'
description = u'All the news from the weblog of F-Secure'
publisher = u'F-Secure'
timefmt = ' [%a, %d %b, %Y]'
encoding = 'ISO-8859-1'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
use_embedded_content = False
language = 'en_EN'
remove_javascript = True
keep_only_tags = [dict(name='div', attrs={'class':'modSectionTd2'})]
remove_tags = [dict(name='a'),dict(name='hr')]
feeds = [(u'Weblog', u'http://www.f-secure.com/weblog/weblog.rss')]
def get_cover_url(self):
return 'http://www.f-secure.com/weblog/archives/images/company_logo.png'

3
recipes/financial_times.recipe
View File

@ -11,7 +11,8 @@ from calibre.web.feeds.news import BasicNewsRecipe
class FinancialTimes(BasicNewsRecipe):
title = u'Financial Times'
__author__ = 'Darko Miletic and Sujata Raman'
description = 'Financial world news'
description = ('Financial world news. Available after 5AM '
'GMT, daily.')
oldest_article = 2
language = 'en'

4
recipes/globe_and_mail.recipe
View File

@ -35,8 +35,8 @@ class AdvancedUserRecipe1287083651(BasicNewsRecipe):
(u'Arts', u'http://www.theglobeandmail.com/news/arts/?service=rss'),
(u'Life', u'http://www.theglobeandmail.com/life/?service=rss'),
(u'Real Estate', u'http://www.theglobeandmail.com/real-estate/?service=rss'),
(u'Auto', u'http://www.theglobeandmail.com/sports/?service=rss'),
(u'Sports', u'http://www.theglobeandmail.com/auto/?service=rss')
(u'Sports', u'http://www.theglobeandmail.com/sports/?service=rss'),
(u'Drive', u'http://www.theglobeandmail.com/auto/?service=rss')
]
preprocess_regexps = [

1
recipes/guardian.recipe
View File

@ -36,6 +36,7 @@ class Guardian(BasicNewsRecipe):
remove_tags = [
dict(name='div', attrs={'class':["video-content","videos-third-column"]}),
dict(name='div', attrs={'id':["article-toolbox","subscribe-feeds",]}),
dict(name='div', attrs={'class':["guardian-tickets promo-component",]}),
dict(name='ul', attrs={'class':["pagination"]}),
dict(name='ul', attrs={'id':["content-actions"]}),
#dict(name='img'),

15
recipes/hawaii.recipe
View File

@ -2,7 +2,7 @@ from calibre.web.feeds.news import BasicNewsRecipe
class AdvancedUserRecipe1282101454(BasicNewsRecipe):
title = 'West Hawaii Today'
__author__ = 'Tony Stegall'
__author__ = 'Tony Stegall, fixed by HK'
language = 'en'
description = 'Westhawaiitoday.com'
publisher = 'West Hawaii '
@ -15,7 +15,14 @@ class AdvancedUserRecipe1282101454(BasicNewsRecipe):
masthead_url = 'http://images.townnews.com/westhawaiitoday.com/art/whttoplogo.gif'
feeds = [ 'http://www.westhawaiitoday.com/rss.xml']
feeds = [
('http://www.westhawaiitoday.com/taxonomy/term/2/feed'), #Local News
('http://www.westhawaiitoday.com/taxonomy/term/15/feed'), #Local Sports
('http://www.westhawaiitoday.com/taxonomy/term/4/feed'), #Local Features
('http://www.westhawaiitoday.com/taxonomy/term/12/feed'), #Obituaries
('http://www.westhawaiitoday.com/taxonomy/term/18/feed'), #Letters
('http://www.westhawaiitoday.com/taxonomy/term/19/feed'), #Editorial
('http://www.westhawaiitoday.com/taxonomy/term/20/feed'), #columns
('http://www.westhawaiitoday.com/taxonomy/term/13/feed') #Volcano Update (Sundays)
]

2
recipes/idnes.recipe
View File

@ -34,7 +34,7 @@ class iHeuteRecipe(BasicNewsRecipe):
dict(name='table', attrs={'class':['video-16ku9']})]
remove_tags_after = [dict(name='div',attrs={'id':['related','related2']})]
keep_only_tags = [dict(name='div', attrs={'class':['art-full adwords-text','dil-day']})
keep_only_tags = [dict(name='div', attrs={'class':['art-full adwords-text','dil-day','art-full']})
,dict(name='table',attrs={'class':['kemel-box']})]
def print_version(self, url):

18
recipes/iht.recipe
View File

@ -15,10 +15,10 @@ class InternationalHeraldTribune(BasicNewsRecipe):
language = 'en'
oldest_article = 1
max_articles_per_feed = 10
max_articles_per_feed = 30
no_stylesheets = True
remove_tags = [dict(name='div', attrs={'class':'footer'}),
remove_tags = [dict(name='div', attrs={'class':['footer','header']}),
dict(name=['form'])]
preprocess_regexps = [
(re.compile(r'<!-- webtrends.*', re.DOTALL),
@ -26,6 +26,8 @@ class InternationalHeraldTribune(BasicNewsRecipe):
]
extra_css = '.headline {font-size: x-large;} \n .fact { padding-top: 10pt }'
remove_empty_feeds = True
feeds = [
(u'Frontpage', u'http://www.iht.com/rss/frontpage.xml'),
(u'Business', u'http://www.iht.com/rss/business.xml'),
@ -46,13 +48,15 @@ class InternationalHeraldTribune(BasicNewsRecipe):
]
temp_files = []
articles_are_obfuscated = True
def get_obfuscated_article(self, url, logger):
masthead_url = 'http://graphics8.nytimes.com/images/misc/iht-masthead-logo.gif'
def get_obfuscated_article(self, url):
br = self.get_browser()
br.open(url)
br.select_form(name='printFriendly')
res = br.submit()
html = res.read()
response1 = br.follow_link(url_regex=re.compile(r'.*pagewanted=print.*'))
html = response1.read()
self.temp_files.append(PersistentTemporaryFile('_iht.html'))
self.temp_files[-1].write(html)
self.temp_files[-1].close()

23
recipes/kommersant.recipe
View File

@ -1,5 +1,5 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2010-2011, Darko Miletic <darko.miletic at gmail.com>'
'''
www.kommersant.ru
'''
@ -20,7 +20,13 @@ class Kommersant_ru(BasicNewsRecipe):
language = 'ru'
publication_type = 'newspaper'
masthead_url = 'http://www.kommersant.ru/CorpPics/logo_daily_1.gif'
extra_css = ' @font-face {font-family: "sans1";src:url(res:///opt/sony/ebook/FONT/tt0003m_.ttf)} body{font-family: Arial, sans1, sans-serif} span#ctl00_ContentPlaceHolderStyle_LabelSubTitle{margin-bottom: 1em; display: block} .author{margin-bottom: 1em; display: block} .paragraph{margin-bottom: 1em; display: block} .vvodka{font-weight: bold; margin-bottom: 1em} '
extra_css = """
@font-face {font-family: "sans1";src:url(res:///opt/sony/ebook/FONT/tt0003m_.ttf)}
body{font-family: Tahoma, Arial, Helvetica, sans1, sans-serif}
.title{font-size: x-large; font-weight: bold; margin-bottom: 1em}
.subtitle{font-size: large; margin-bottom: 1em}
.document_vvodka{font-weight: bold; margin-bottom: 1em}
"""
conversion_options = {
'comment' : description
@ -29,14 +35,11 @@ class Kommersant_ru(BasicNewsRecipe):
, 'language' : language
}
keep_only_tags = [
dict(attrs={'id':'ctl00_ContentPlaceHolderStyle_PanelHeader'})
,dict(attrs={'class':['vvodka','paragraph','author']})
]
remove_tags = [dict(name=['iframe','object','link','img','base'])]
keep_only_tags = [dict(attrs={'class':['document','document_vvodka','document_text','document_authors vblock']})]
remove_tags = [dict(name=['iframe','object','link','img','base','meta'])]
feeds = [(u'Articles', u'http://feeds.kommersant.ru/RSS_Export/RU/daily.xml')]
def print_version(self, url):
return url.replace('doc-rss.aspx','doc.aspx') + '&print=true'
def print_version(self, url):
return url.replace('/doc-rss/','/Doc/') + '/Print'

6
recipes/perfil.recipe
View File

@ -1,5 +1,5 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2010-2011, Darko Miletic <darko.miletic at gmail.com>'
'''
perfil.com
'''
@ -39,9 +39,9 @@ class Perfil(BasicNewsRecipe):
dict(name=['iframe','embed','object','base','meta','link'])
,dict(name='a', attrs={'href':'#comentarios'})
,dict(name='div', attrs={'class':'foto3'})
,dict(name='img', attrs={'alt':'ampliar'})
,dict(name='img', attrs={'alt':['ampliar','Ampliar']})
]
keep_only_tags=[dict(attrs={'class':['bd468a','cuerpoSuperior']})]
keep_only_tags=[dict(attrs={'class':['articulo','cuerpoSuperior']})]
remove_attributes=['onload','lang','width','height','border']
feeds = [

1
recipes/smith.recipe
View File

@ -7,6 +7,7 @@ class SmithsonianMagazine(BasicNewsRecipe):
__author__ = 'Krittika Goyal'
oldest_article = 31#days
max_articles_per_feed = 50
use_embedded_content = False
#encoding = 'latin1'
recursions = 1
match_regexps = ['&page=[2-9]$']

29
recipes/toi.recipe
View File

@ -1,3 +1,4 @@
import re
from calibre.web.feeds.news import BasicNewsRecipe
class TimesOfIndia(BasicNewsRecipe):
@ -8,10 +9,10 @@ class TimesOfIndia(BasicNewsRecipe):
max_articles_per_feed = 25
no_stylesheets = True
keep_only_tags = [dict(attrs={'class':'maintable12'})]
keep_only_tags = [{'class':['maintable12', 'prttabl']}]
remove_tags = [
dict(style=lambda x: x and 'float' in x),
dict(attrs={'class':'prvnxtbg'}),
{'class':['prvnxtbg', 'footbdrin', 'bcclftr']},
]
feeds = [
@ -38,8 +39,28 @@ class TimesOfIndia(BasicNewsRecipe):
('Most Read',
'http://timesofindia.indiatimes.com/rssfeedmostread.cms')
]
def print_version(self, url):
return url + '?prtpage=1'
def get_article_url(self, article):
url = BasicNewsRecipe.get_article_url(self, article)
if '/0Ltimesofindia' in url:
url = url.partition('/0L')[-1]
url = url.replace('0B', '.').replace('0N', '.com').replace('0C',
'/').replace('0E', '-')
url = 'http://' + url.rpartition('/')[0]
match = re.search(r'/([0-9a-zA-Z]+?)\.cms', url)
if match is not None:
num = match.group(1)
num = re.sub(r'[^0-9]', '', num)
return ('http://timesofindia.indiatimes.com/articleshow/%s.cms?prtpage=1' %
num)
else:
cms = re.search(r'/(\d+)\.cms', url)
if cms is not None:
return ('http://timesofindia.indiatimes.com/articleshow/%s.cms?prtpage=1' %
cms.group(1))
return url
def preprocess_html(self, soup):
return soup

1
setup/publish.py
View File

@ -45,7 +45,6 @@ class Stage3(Command):
sub_commands = ['upload_user_manual', 'upload_demo', 'sdist',
'upload_to_sourceforge', 'upload_to_google_code',
'tag_release', 'upload_to_server',
'upload_to_mobileread',
]
class Stage4(Command):

28
src/calibre/__init__.py
View File

@ -217,14 +217,25 @@ def filename_to_utf8(name):
return name.decode(codec, 'replace').encode('utf8')
def extract(path, dir):
ext = os.path.splitext(path)[1][1:].lower()
extractor = None
if ext in ['zip', 'cbz', 'epub', 'oebzip']:
from calibre.libunzip import extract as zipextract
extractor = zipextract
elif ext in ['cbr', 'rar']:
# First use the file header to identify its type
with open(path, 'rb') as f:
id_ = f.read(3)
if id_ == b'Rar':
from calibre.libunrar import extract as rarextract
extractor = rarextract
elif id_.startswith(b'PK'):
from calibre.libunzip import extract as zipextract
extractor = zipextract
if extractor is None:
# Fallback to file extension
ext = os.path.splitext(path)[1][1:].lower()
if ext in ['zip', 'cbz', 'epub', 'oebzip']:
from calibre.libunzip import extract as zipextract
extractor = zipextract
elif ext in ['cbr', 'rar']:
from calibre.libunrar import extract as rarextract
extractor = rarextract
if extractor is None:
raise Exception('Unknown archive type')
extractor(path, dir)
@ -281,16 +292,17 @@ def get_parsed_proxy(typ='http', debug=True):
def random_user_agent():
choices = [
'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/523.15 (KHTML, like Gecko, Safari/419.3) Arora/0.3 (Change: 287 c9dfb30)',
'Mozilla/5.0 (Windows NT 5.2; rv:2.0.1) Gecko/20100101 Firefox/4.0.1',
'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1',
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1',
'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11',
'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.153.1 Safari/525.19',
'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11',
'Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en; rv:1.8.1.14) Gecko/20080409 Camino/1.6 (like Firefox/2.0.0.14)',
'Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.1) Gecko/20060118 Camino/1.0b2+',
'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3',
'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.78 Safari/532.5',
'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)',
]
#return choices[-1]
return choices[random.randint(0, len(choices)-1)]

2
src/calibre/constants.py
View File

@ -2,7 +2,7 @@ __license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'
__appname__ = 'calibre'
__version__ = '0.7.53'
__version__ = '0.7.54'
__author__ = "Kovid Goyal <kovid@kovidgoyal.net>"
import re, importlib

80
src/calibre/customize/builtins.py
View File

@ -10,6 +10,7 @@ from calibre.constants import numeric_version
from calibre.ebooks.metadata.archive import ArchiveExtract, get_cbz_metadata
from calibre.ebooks.metadata.opf2 import metadata_to_opf
from calibre.ebooks.oeb.base import OEB_IMAGES
from calibre.utils.config import test_eight_code
# To archive plugins {{{
class HTML2ZIP(FileTypePlugin):
@ -166,6 +167,14 @@ class ComicMetadataReader(MetadataReaderPlugin):
description = _('Extract cover from comic files')
def get_metadata(self, stream, ftype):
if hasattr(stream, 'seek') and hasattr(stream, 'tell'):
pos = stream.tell()
id_ = stream.read(3)
stream.seek(pos)
if id_ == b'Rar':
ftype = 'cbr'
elif id.startswith(b'PK'):
ftype = 'cbz'
if ftype == 'cbr':
from calibre.libunrar import extract_first_alphabetically as extract_first
extract_first
@ -231,6 +240,17 @@ class HTMLMetadataReader(MetadataReaderPlugin):
from calibre.ebooks.metadata.html import get_metadata
return get_metadata(stream)
class HTMLZMetadataReader(MetadataReaderPlugin):
name = 'Read HTMLZ metadata'
file_types = set(['htmlz'])
description = _('Read metadata from %s files') % 'HTMLZ'
author = 'John Schember'
def get_metadata(self, stream, ftype):
from calibre.ebooks.metadata.extz import get_metadata
return get_metadata(stream)
class IMPMetadataReader(MetadataReaderPlugin):
name = 'Read IMP metadata'
@ -407,7 +427,7 @@ class TXTZMetadataReader(MetadataReaderPlugin):
author = 'John Schember'
def get_metadata(self, stream, ftype):
from calibre.ebooks.metadata.txtz import get_metadata
from calibre.ebooks.metadata.extz import get_metadata
return get_metadata(stream)
class ZipMetadataReader(MetadataReaderPlugin):
@ -433,6 +453,17 @@ class EPUBMetadataWriter(MetadataWriterPlugin):
from calibre.ebooks.metadata.epub import set_metadata
set_metadata(stream, mi, apply_null=self.apply_null)
class HTMLZMetadataWriter(MetadataWriterPlugin):
name = 'Set HTMLZ metadata'
file_types = set(['htmlz'])
description = _('Set metadata from %s files') % 'HTMLZ'
author = 'John Schember'
def set_metadata(self, stream, mi, type):
from calibre.ebooks.metadata.extz import set_metadata
set_metadata(stream, mi)
class LRFMetadataWriter(MetadataWriterPlugin):
name = 'Set LRF metadata'
@ -505,7 +536,7 @@ class TXTZMetadataWriter(MetadataWriterPlugin):
author = 'John Schember'
def set_metadata(self, stream, mi, type):
from calibre.ebooks.metadata.txtz import set_metadata
from calibre.ebooks.metadata.extz import set_metadata
set_metadata(stream, mi)
# }}}
@ -514,6 +545,7 @@ from calibre.ebooks.comic.input import ComicInput
from calibre.ebooks.epub.input import EPUBInput
from calibre.ebooks.fb2.input import FB2Input
from calibre.ebooks.html.input import HTMLInput
from calibre.ebooks.htmlz.input import HTMLZInput
from calibre.ebooks.lit.input import LITInput
from calibre.ebooks.mobi.input import MOBIInput
from calibre.ebooks.odt.input import ODTInput
@ -544,6 +576,7 @@ from calibre.ebooks.tcr.output import TCROutput
from calibre.ebooks.txt.output import TXTOutput
from calibre.ebooks.txt.output import TXTZOutput
from calibre.ebooks.html.output import HTMLOutput
from calibre.ebooks.htmlz.output import HTMLZOutput
from calibre.ebooks.snb.output import SNBOutput
from calibre.customize.profiles import input_profiles, output_profiles
@ -580,25 +613,40 @@ from calibre.devices.folder_device.driver import FOLDER_DEVICE_FOR_CONFIG
from calibre.devices.kobo.driver import KOBO
from calibre.devices.bambook.driver import BAMBOOK
from calibre.ebooks.metadata.fetch import GoogleBooks, ISBNDB, Amazon, \
KentDistrictLibrary
from calibre.ebooks.metadata.douban import DoubanBooks
from calibre.ebooks.metadata.nicebooks import NiceBooks, NiceBooksCovers
from calibre.ebooks.metadata.covers import OpenLibraryCovers, \
AmazonCovers, DoubanCovers
from calibre.library.catalog import CSV_XML, EPUB_MOBI, BIBTEX
from calibre.ebooks.epub.fix.unmanifested import Unmanifested
from calibre.ebooks.epub.fix.epubcheck import Epubcheck
plugins = [HTML2ZIP, PML2PMLZ, TXT2TXTZ, ArchiveExtract, GoogleBooks, ISBNDB, Amazon,
KentDistrictLibrary, DoubanBooks, NiceBooks, CSV_XML, EPUB_MOBI, BIBTEX, Unmanifested,
Epubcheck, OpenLibraryCovers, AmazonCovers, DoubanCovers,
NiceBooksCovers]
plugins = [HTML2ZIP, PML2PMLZ, TXT2TXTZ, ArchiveExtract, CSV_XML, EPUB_MOBI, BIBTEX, Unmanifested,
Epubcheck, ]
if test_eight_code:
# New metadata download plugins {{{
from calibre.ebooks.metadata.sources.google import GoogleBooks
from calibre.ebooks.metadata.sources.amazon import Amazon
from calibre.ebooks.metadata.sources.openlibrary import OpenLibrary
plugins += [GoogleBooks, Amazon, OpenLibrary]
# }}}
else:
from calibre.ebooks.metadata.fetch import GoogleBooks, ISBNDB, Amazon, \
KentDistrictLibrary
from calibre.ebooks.metadata.douban import DoubanBooks
from calibre.ebooks.metadata.nicebooks import NiceBooks, NiceBooksCovers
from calibre.ebooks.metadata.covers import OpenLibraryCovers, \
AmazonCovers, DoubanCovers
plugins += [GoogleBooks, ISBNDB, Amazon,
OpenLibraryCovers, AmazonCovers, DoubanCovers,
NiceBooksCovers, KentDistrictLibrary, DoubanBooks, NiceBooks]
plugins += [
ComicInput,
EPUBInput,
FB2Input,
HTMLInput,
HTMLZInput,
LITInput,
MOBIInput,
ODTInput,
@ -630,6 +678,7 @@ plugins += [
TXTOutput,
TXTZOutput,
HTMLOutput,
HTMLZOutput,
SNBOutput,
]
# Order here matters. The first matched device is the one used.
@ -1029,11 +1078,4 @@ plugins += [LookAndFeel, Behavior, Columns, Toolbar, Search, InputOptions,
#}}}
# New metadata download plugins {{{
from calibre.ebooks.metadata.sources.google import GoogleBooks
from calibre.ebooks.metadata.sources.amazon import Amazon
from calibre.ebooks.metadata.sources.openlibrary import OpenLibrary
plugins += [GoogleBooks, Amazon, OpenLibrary]
# }}}

4
src/calibre/customize/profiles.py
View File

@ -471,8 +471,8 @@ class KoboReaderOutput(OutputProfile):
description = _('This profile is intended for the Kobo Reader.')
screen_size = (540, 718)
comic_screen_size = (540, 718)
screen_size = (536, 710)
comic_screen_size = (536, 710)
dpi = 168.451
fbase = 12
fsizes = [7.5, 9, 10, 12, 15.5, 20, 22, 24]

7
src/calibre/devices/android/driver.py
View File

@ -36,7 +36,9 @@ class ANDROID(USBMS):
# Motorola
0x22b8 : { 0x41d9 : [0x216], 0x2d61 : [0x100], 0x2d67 : [0x100],
0x41db : [0x216], 0x4285 : [0x216], 0x42a3 : [0x216],
0x4286 : [0x216], 0x42b3 : [0x216], 0x42b4 : [0x216] },
0x4286 : [0x216], 0x42b3 : [0x216], 0x42b4 : [0x216],
0x7086 : [0x0226],
},
# Sony Ericsson
0xfce : { 0xd12e : [0x0100]},
@ -101,7 +103,8 @@ class ANDROID(USBMS):
'SCH-I500_CARD', 'SPH-D700_CARD', 'MB810', 'GT-P1000', 'DESIRE',
'SGH-T849', '_MB300', 'A70S', 'S_ANDROID', 'A101IT', 'A70H',
'IDEOS_TABLET', 'MYTOUCH_4G', 'UMS_COMPOSITE', 'SCH-I800_CARD',
'7', 'A956', 'A955', 'A43', 'ANDROID_PLATFORM', 'TEGRA_2']
'7', 'A956', 'A955', 'A43', 'ANDROID_PLATFORM', 'TEGRA_2',
'MB860']
WINDOWS_CARD_A_MEM = ['ANDROID_PHONE', 'GT-I9000_CARD', 'SGH-I897',
'FILE-STOR_GADGET', 'SGH-T959', 'SAMSUNG_ANDROID', 'GT-P1000_CARD',
'A70S', 'A101IT', '7']

3
src/calibre/devices/eb600/driver.py
View File

@ -244,7 +244,8 @@ class POCKETBOOK602(USBMS):
BCD = [0x0324]
VENDOR_NAME = ''
WINDOWS_MAIN_MEM = WINDOWS_CARD_A_MEM = ['PB602', 'PB603', 'PB902', 'PB903']
WINDOWS_MAIN_MEM = WINDOWS_CARD_A_MEM = ['PB602', 'PB603', 'PB902',
'PB903', 'PB']
class POCKETBOOK701(USBMS):

13
src/calibre/ebooks/chardet/__init__.py
View File

@ -100,6 +100,12 @@ def xml_to_unicode(raw, verbose=False, strip_encoding_pats=False,
try:
if encoding.lower().strip() == 'macintosh':
encoding = 'mac-roman'
if encoding.lower().replace('_', '-').strip() in (
'gb2312', 'chinese', 'csiso58gb231280', 'euc-cn', 'euccn',
'eucgb2312-cn', 'gb2312-1980', 'gb2312-80', 'iso-ir-58'):
# Microsoft Word exports to HTML with encoding incorrectly set to
# gb2312 instead of gbk. gbk is a superset of gb2312, anyway.
encoding = 'gbk'
raw = raw.decode(encoding, 'replace')
except LookupError:
encoding = 'utf-8'
@ -110,11 +116,6 @@ def xml_to_unicode(raw, verbose=False, strip_encoding_pats=False,
if resolve_entities:
raw = substitute_entites(raw)
if encoding and encoding.lower().replace('_', '-').strip() in (
'gb2312', 'chinese', 'csiso58gb231280', 'euc-cn', 'euccn',
'eucgb2312-cn', 'gb2312-1980', 'gb2312-80', 'iso-ir-58'):
# Microsoft Word exports to HTML with encoding incorrectly set to
# gb2312 instead of gbk. gbk is a superset of gb2312, anyway.
encoding = 'gbk'
return raw, encoding

0
src/calibre/ebooks/htmlz/__init__.py Normal file
View File

66
src/calibre/ebooks/htmlz/input.py Normal file
View File

@ -0,0 +1,66 @@
# -*- coding: utf-8 -*-
from __future__ import (unicode_literals, division, absolute_import, print_function)
__license__ = 'GPL 3'
__copyright__ = '2011, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
import os
from calibre import walk
from calibre.customize.conversion import InputFormatPlugin
from calibre.utils.zipfile import ZipFile
class HTMLZInput(InputFormatPlugin):
name = 'HTLZ Input'
author = 'John Schember'
description = 'Convert HTML files to HTML'
file_types = set(['htmlz'])
def convert(self, stream, options, file_ext, log,
accelerators):
self.log = log
html = u''
# Extract content from zip archive.
zf = ZipFile(stream)
zf.extractall('.')
for x in walk('.'):
if os.path.splitext(x)[1].lower() in ('.html', '.xhtml', '.htm'):
with open(x, 'rb') as tf:
html = tf.read()
break
# Run the HTML through the html processing plugin.
from calibre.customize.ui import plugin_for_input_format
html_input = plugin_for_input_format('html')
for opt in html_input.options:
setattr(options, opt.option.name, opt.recommended_value)
options.input_encoding = 'utf-8'
base = os.getcwdu()
fname = os.path.join(base, 'index.html')
c = 0
while os.path.exists(fname):
c += 1
fname = 'index%d.html'%c
htmlfile = open(fname, 'wb')
with htmlfile:
htmlfile.write(html.encode('utf-8'))
odi = options.debug_pipeline
options.debug_pipeline = None
# Generate oeb from html conversion.
oeb = html_input.convert(open(htmlfile.name, 'rb'), options, 'html', log,
{})
options.debug_pipeline = odi
os.remove(htmlfile.name)
# Set metadata from file.
from calibre.customize.ui import get_file_type_metadata
from calibre.ebooks.oeb.transforms.metadata import meta_info_to_oeb_metadata
mi = get_file_type_metadata(stream, file_ext)
meta_info_to_oeb_metadata(mi, oeb.metadata, log)
return oeb

385
src/calibre/ebooks/htmlz/oeb2html.py Normal file
View File

@ -0,0 +1,385 @@
# -*- coding: utf-8 -*-
from __future__ import (unicode_literals, division, absolute_import, print_function)
__license__ = 'GPL 3'
__copyright__ = '2011, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
'''
Transform OEB content into a single (more or less) HTML file.
'''
import os
from functools import partial
from lxml import html
from urlparse import urldefrag
from calibre import prepare_string_for_xml
from calibre.ebooks.oeb.base import XHTML, XHTML_NS, barename, namespace,\
OEB_IMAGES, XLINK, rewrite_links
from calibre.ebooks.oeb.stylizer import Stylizer
from calibre.utils.logging import default_log
class OEB2HTML(object):
'''
Base class. All subclasses should implement dump_text to actually transform
content. Also, callers should use oeb2html to get the transformed html.
links and images can be retrieved after calling oeb2html to get the mapping
of OEB links and images to the new names used in the html returned by oeb2html.
Images will always be referenced as if they are in an images directory.
Use get_css to get the CSS classes for the OEB document as a string.
'''
def __init__(self, log=None):
self.log = default_log if log is None else log
self.links = {}
self.images = {}
def oeb2html(self, oeb_book, opts):
self.log.info('Converting OEB book to HTML...')
self.opts = opts
self.links = {}
self.images = {}
self.base_hrefs = [item.href for item in oeb_book.spine]
self.map_resources(oeb_book)
return self.mlize_spine(oeb_book)
def mlize_spine(self, oeb_book):
output = [u'<html><body><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8" /></head>']
for item in oeb_book.spine:
self.log.debug('Converting %s to HTML...' % item.href)
self.rewrite_ids(item.data, item)
rewrite_links(item.data, partial(self.rewrite_link, page=item))
stylizer = Stylizer(item.data, item.href, oeb_book, self.opts)
output += self.dump_text(item.data.find(XHTML('body')), stylizer, item)
output.append('\n\n')
output.append('</body></html>')
return ''.join(output)
def dump_text(self, elem, stylizer, page):
raise NotImplementedError
def get_link_id(self, href, id=''):
if id:
href += '#%s' % id
if href not in self.links:
self.links[href] = '#calibre_link-%s' % len(self.links.keys())
return self.links[href]
def map_resources(self, oeb_book):
for item in oeb_book.manifest:
if item.media_type in OEB_IMAGES:
if item.href not in self.images:
ext = os.path.splitext(item.href)[1]
fname = '%s%s' % (len(self.images), ext)
fname = fname.zfill(10)
self.images[item.href] = fname
if item in oeb_book.spine:
self.get_link_id(item.href)
root = item.data.find(XHTML('body'))
link_attrs = set(html.defs.link_attrs)
link_attrs.add(XLINK('href'))
for el in root.iter():
attribs = el.attrib
try:
if not isinstance(el.tag, basestring):
continue
except:
continue
for attr in attribs:
if attr in link_attrs:
href = item.abshref(attribs[attr])
href, id = urldefrag(href)
if href in self.base_hrefs:
self.get_link_id(href, id)
def rewrite_link(self, url, page=None):
if not page:
return url
abs_url = page.abshref(url)
if abs_url in self.images:
return 'images/%s' % self.images[abs_url]
if abs_url in self.links:
return self.links[abs_url]
return url
def rewrite_ids(self, root, page):
for el in root.iter():
try:
tag = el.tag
except UnicodeDecodeError:
continue
if tag == XHTML('body'):
el.attrib['id'] = self.get_link_id(page.href)[1:]
continue
if 'id' in el.attrib:
el.attrib['id'] = self.get_link_id(page.href, el.attrib['id'])[1:]
def get_css(self, oeb_book):
css = u''
for item in oeb_book.manifest:
if item.media_type == 'text/css':
css = item.data.cssText
break
return css
class OEB2HTMLNoCSSizer(OEB2HTML):
'''
This will remap a small number of CSS styles to equivalent HTML tags.
'''
def dump_text(self, elem, stylizer, page):
'''
@elem: The element in the etree that we are working on.
@stylizer: The style information attached to the element.
'''
# We can only processes tags. If there isn't a tag return any text.
if not isinstance(elem.tag, basestring) \
or namespace(elem.tag) != XHTML_NS:
p = elem.getparent()
if p is not None and isinstance(p.tag, basestring) and namespace(p.tag) == XHTML_NS \
and elem.tail:
return [elem.tail]
return ['']
# Setup our variables.
text = ['']
style = stylizer.style(elem)
tags = []
tag = barename(elem.tag)
attribs = elem.attrib
if tag == 'body':
tag = 'div'
tags.append(tag)
# Ignore anything that is set to not be displayed.
if style['display'] in ('none', 'oeb-page-head', 'oeb-page-foot') \
or style['visibility'] == 'hidden':
return ['']
# Remove attributes we won't want.
if 'class' in attribs:
del attribs['class']
if 'style' in attribs:
del attribs['style']
# Turn the rest of the attributes into a string we can write with the tag.
at = ''
for k, v in attribs.items():
at += ' %s="%s"' % (k, prepare_string_for_xml(v, attribute=True))
# Write the tag.
text.append('<%s%s>' % (tag, at))
# Turn styles into tags.
if style['font-weight'] in ('bold', 'bolder'):
text.append('<b>')
tags.append('b')
if style['font-style'] == 'italic':
text.append('<i>')
tags.append('i')
if style['text-decoration'] == 'underline':
text.append('<u>')
tags.append('u')
if style['text-decoration'] == 'line-through':
text.append('<s>')
tags.append('s')
# Process tags that contain text.
if hasattr(elem, 'text') and elem.text:
text.append(elem.text)
# Recurse down into tags within the tag we are in.
for item in elem:
text += self.dump_text(item, stylizer, page)
# Close all open tags.
tags.reverse()
for t in tags:
text.append('</%s>' % t)
# Add the text that is outside of the tag.
if hasattr(elem, 'tail') and elem.tail:
text.append(elem.tail)
return text
class OEB2HTMLInlineCSSizer(OEB2HTML):
'''
Turns external CSS classes into inline style attributes.
'''
def dump_text(self, elem, stylizer, page):
'''
@elem: The element in the etree that we are working on.
@stylizer: The style information attached to the element.
'''
# We can only processes tags. If there isn't a tag return any text.
if not isinstance(elem.tag, basestring) \
or namespace(elem.tag) != XHTML_NS:
p = elem.getparent()
if p is not None and isinstance(p.tag, basestring) and namespace(p.tag) == XHTML_NS \
and elem.tail:
return [elem.tail]
return ['']
# Setup our variables.
text = ['']
style = stylizer.style(elem)
tags = []
tag = barename(elem.tag)
attribs = elem.attrib
style_a = '%s' % style
if tag == 'body':
tag = 'div'
if not style['page-break-before'] == 'always':
style_a = 'page-break-before: always;' + ' ' if style_a else '' + style_a
tags.append(tag)
# Remove attributes we won't want.
if 'class' in attribs:
del attribs['class']
if 'style' in attribs:
del attribs['style']
# Turn the rest of the attributes into a string we can write with the tag.
at = ''
for k, v in attribs.items():
at += ' %s="%s"' % (k, prepare_string_for_xml(v, attribute=True))
# Turn style into strings for putting in the tag.
style_t = ''
if style_a:
style_t = ' style="%s"' % style_a
# Write the tag.
text.append('<%s%s%s>' % (tag, at, style_t))
# Process tags that contain text.
if hasattr(elem, 'text') and elem.text:
text.append(elem.text)
# Recurse down into tags within the tag we are in.
for item in elem:
text += self.dump_text(item, stylizer, page)
# Close all open tags.
tags.reverse()
for t in tags:
text.append('</%s>' % t)
# Add the text that is outside of the tag.
if hasattr(elem, 'tail') and elem.tail:
text.append(elem.tail)
return text
class OEB2HTMLClassCSSizer(OEB2HTML):
'''
Use CSS classes. css_style option can specify whether to use
inline classes (style tag in the head) or reference an external
CSS file called style.css.
'''
def mlize_spine(self, oeb_book):
output = []
for item in oeb_book.spine:
self.log.debug('Converting %s to HTML...' % item.href)
self.rewrite_ids(item.data, item)
rewrite_links(item.data, partial(self.rewrite_link, page=item))
stylizer = Stylizer(item.data, item.href, oeb_book, self.opts)
output += self.dump_text(item.data.find(XHTML('body')), stylizer, item)
output.append('\n\n')
if self.opts.htmlz_class_style == 'external':
css = u'<link href="style.css" rel="stylesheet" type="text/css" />'
else:
css = u'<style type="text/css">' + self.get_css(oeb_book) + u'</style>'
output = [u'<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8" />'] + [css] + [u'</head><body>'] + output + [u'</body></html>']
return ''.join(output)
def dump_text(self, elem, stylizer, page):
'''
@elem: The element in the etree that we are working on.
@stylizer: The style information attached to the element.
'''
# We can only processes tags. If there isn't a tag return any text.
if not isinstance(elem.tag, basestring) \
or namespace(elem.tag) != XHTML_NS:
p = elem.getparent()
if p is not None and isinstance(p.tag, basestring) and namespace(p.tag) == XHTML_NS \
and elem.tail:
return [elem.tail]
return ['']
# Setup our variables.
text = ['']
tags = []
tag = barename(elem.tag)
attribs = elem.attrib
if tag == 'body':
tag = 'div'
tags.append(tag)
# Remove attributes we won't want.
if 'style' in attribs:
del attribs['style']
# Turn the rest of the attributes into a string we can write with the tag.
at = ''
for k, v in attribs.items():
at += ' %s="%s"' % (k, prepare_string_for_xml(v, attribute=True))
# Write the tag.
text.append('<%s%s>' % (tag, at))
# Process tags that contain text.
if hasattr(elem, 'text') and elem.text:
text.append(elem.text)
# Recurse down into tags within the tag we are in.
for item in elem:
text += self.dump_text(item, stylizer, page)
# Close all open tags.
tags.reverse()
for t in tags:
text.append('</%s>' % t)
# Add the text that is outside of the tag.
if hasattr(elem, 'tail') and elem.tail:
text.append(elem.tail)
return text
def oeb2html_no_css(oeb_book, log, opts):
izer = OEB2HTMLNoCSSizer(log)
html = izer.oeb2html(oeb_book, opts)
images = izer.images
return (html, images)
def oeb2html_inline_css(oeb_book, log, opts):
izer = OEB2HTMLInlineCSSizer(log)
html = izer.oeb2html(oeb_book, opts)
images = izer.images
return (html, images)
def oeb2html_class_css(oeb_book, log, opts):
izer = OEB2HTMLClassCSSizer(log)
setattr(opts, 'class_style', 'inline')
html = izer.oeb2html(oeb_book, opts)
images = izer.images
return (html, images)

83
src/calibre/ebooks/htmlz/output.py Normal file
View File

@ -0,0 +1,83 @@
# -*- coding: utf-8 -*-
from __future__ import (unicode_literals, division, absolute_import, print_function)
__license__ = 'GPL 3'
__copyright__ = '2011, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
import os
from lxml import etree
from calibre.customize.conversion import OutputFormatPlugin, \
OptionRecommendation
from calibre.ebooks.oeb.base import OEB_IMAGES
from calibre.ptempfile import TemporaryDirectory
from calibre.utils.zipfile import ZipFile
class HTMLZOutput(OutputFormatPlugin):
name = 'HTMLZ Output'
author = 'John Schember'
file_type = 'htmlz'
options = set([
OptionRecommendation(name='htmlz_css_type', recommended_value='class',
level=OptionRecommendation.LOW,
choices=['class', 'inline', 'tag'],
help=_('Specify the handling of CSS. Default is class.\n'
'class: Use CSS classes and have elements reference them.\n'
'inline: Write the CSS as an inline style attribute.\n'
'tag: Turn as many CSS styles as possible into HTML tags.'
)),
OptionRecommendation(name='htmlz_class_style', recommended_value='external',
level=OptionRecommendation.LOW,
choices=['external', 'inline'],
help=_('How to handle the CSS when using css-type = \'class\'.\n'
'Default is external.\n'
'external: Use an external CSS file that is linked in the document.\n'
'inline: Place the CSS in the head section of the document.'
)),
])
def convert(self, oeb_book, output_path, input_plugin, opts, log):
# HTML
if opts.htmlz_css_type == 'inline':
from calibre.ebooks.htmlz.oeb2html import OEB2HTMLInlineCSSizer
OEB2HTMLizer = OEB2HTMLInlineCSSizer
elif opts.htmlz_css_type == 'tag':
from calibre.ebooks.htmlz.oeb2html import OEB2HTMLNoCSSizer
OEB2HTMLizer = OEB2HTMLNoCSSizer
else:
from calibre.ebooks.htmlz.oeb2html import OEB2HTMLClassCSSizer as OEB2HTMLizer
with TemporaryDirectory('_htmlz_output') as tdir:
htmlizer = OEB2HTMLizer(log)
html = htmlizer.oeb2html(oeb_book, opts)
with open(os.path.join(tdir, 'index.html'), 'wb') as tf:
tf.write(html)
# CSS
if opts.htmlz_css_type == 'class' and opts.htmlz_class_style == 'external':
with open(os.path.join(tdir, 'style.css'), 'wb') as tf:
tf.write(htmlizer.get_css(oeb_book))
# Images
images = htmlizer.images
if images:
if not os.path.exists(os.path.join(tdir, 'images')):
os.makedirs(os.path.join(tdir, 'images'))
for item in oeb_book.manifest:
if item.media_type in OEB_IMAGES and item.href in images:
fname = os.path.join(tdir, 'images', images[item.href])
with open(fname, 'wb') as img:
img.write(item.data)
# Metadata
with open(os.path.join(tdir, 'metadata.opf'), 'wb') as mdataf:
mdataf.write(etree.tostring(oeb_book.metadata.to_opf1()))
htmlz = ZipFile(output_path, 'w')
htmlz.add_dir(tdir)

10
src/calibre/ebooks/metadata/book/base.py
View File

@ -125,7 +125,10 @@ class Metadata(object):
_data = object.__getattribute__(self, '_data')
if field in TOP_LEVEL_IDENTIFIERS:
field, val = self._clean_identifier(field, val)
_data['identifiers'].update({field: val})
identifiers = _data['identifiers']
identifiers.pop(field, None)
if val:
identifiers[field] = val
elif field == 'identifiers':
if not val:
val = copy.copy(NULL_VALUES.get('identifiers', None))
@ -224,8 +227,7 @@ class Metadata(object):
identifiers = object.__getattribute__(self,
'_data')['identifiers']
if not val and typ in identifiers:
identifiers.pop(typ)
identifiers.pop(typ, None)
if val:
identifiers[typ] = val
@ -647,7 +649,7 @@ class Metadata(object):
fmt('Tags', u', '.join([unicode(t) for t in self.tags]))
if self.series:
fmt('Series', self.series + ' #%s'%self.format_series_index())
if self.language:
if not self.is_null('language'):
fmt('Language', self.language)
if self.rating is not None:
fmt('Rating', self.rating)

2
src/calibre/ebooks/metadata/txtz.py → src/calibre/ebooks/metadata/extz.py
View File

@ -4,7 +4,7 @@ __license__ = 'GPL v3'
__copyright__ = '2011, John Schember <john@nachtimwald.com>'
'''
Read meta information from TXT files
Read meta information from extZ (TXTZ, HTMLZ...) files.
'''
import os

1
src/calibre/ebooks/metadata/google_books.py
View File

@ -193,6 +193,7 @@ class ResultList(list):
def search(title=None, author=None, publisher=None, isbn=None,
min_viewability='none', verbose=False, max_results=40):
br = browser()
br.set_handle_gzip(True)
start, entries = 1, []
while start > 0 and len(entries) <= max_results:
new, start = Query(title=title, author=author, publisher=publisher,

26
src/calibre/ebooks/metadata/sources/amazon.py
View File

@ -23,7 +23,7 @@ from calibre.ebooks.metadata.book.base import Metadata
from calibre.library.comments import sanitize_comments_html
from calibre.utils.date import parse_date
class Worker(Thread): # {{{
class Worker(Thread): # Get details {{{
'''
Get book details from amazons book page in a separate thread
@ -64,7 +64,7 @@ class Worker(Thread): # {{{
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
# open('/t/t.html', 'wb').write(raw)
#open('/t/t.html', 'wb').write(raw)
if '<title>404 - ' in raw:
self.log.error('URL malformed: %r'%self.url)
@ -218,6 +218,9 @@ class Worker(Thread): # {{{
' @class="emptyClear" or @href]'):
c.getparent().remove(c)
desc = tostring(desc, method='html', encoding=unicode).strip()
# Encoding bug in Amazon data U+fffd (replacement char)
# in some examples it is present in place of '
desc = desc.replace('\ufffd', "'")
# remove all attributes from tags
desc = re.sub(r'<([a-zA-Z0-9]+)\s[^>]+>', r'<\1>', desc)
# Collapse whitespace
@ -276,12 +279,14 @@ class Worker(Thread): # {{{
class Amazon(Source):
name = 'Amazon'
name = 'Amazon Store'
description = _('Downloads metadata from Amazon')
capabilities = frozenset(['identify', 'cover'])
touched_fields = frozenset(['title', 'authors', 'identifier:amazon',
'identifier:isbn', 'rating', 'comments', 'publisher', 'pubdate'])
has_html_comments = True
supports_gzip_transfer_encoding = True
AMAZON_DOMAINS = {
'com': _('US'),
@ -408,6 +413,18 @@ class Amazon(Source):
if 'bulk pack' not in title:
matches.append(a.get('href'))
break
if not matches:
# This can happen for some user agents that Amazon thinks are
# mobile/less capable
log('Trying alternate results page markup')
for td in root.xpath(
r'//div[@id="Results"]/descendant::td[starts-with(@id, "search:Td:")]'):
for a in td.xpath(r'descendant::td[@class="dataColumn"]/descendant::a[@href]/span[@class="srTitle"]/..'):
title = tostring(a, method='text', encoding=unicode).lower()
if 'bulk pack' not in title:
matches.append(a.get('href'))
break
# Keep only the top 5 matches as the matches are sorted by relevance by
# Amazon so lower matches are not likely to be very relevant
@ -476,9 +493,10 @@ class Amazon(Source):
if abort.is_set():
return
br = self.browser
log('Downloading cover from:', cached_url)
try:
cdata = br.open_novisit(cached_url, timeout=timeout).read()
result_queue.put(cdata)
result_queue.put((self, cdata))
except:
log.exception('Failed to download cover from:', cached_url)
# }}}

73
src/calibre/ebooks/metadata/sources/base.py
View File

@ -15,9 +15,20 @@ from calibre.customize import Plugin
from calibre.utils.logging import ThreadSafeLog, FileStream
from calibre.utils.config import JSONConfig
from calibre.utils.titlecase import titlecase
from calibre.utils.icu import capitalize, lower
from calibre.ebooks.metadata import check_isbn
msprefs = JSONConfig('metadata_sources.json')
msprefs = JSONConfig('metadata_sources/global.json')
msprefs.defaults['txt_comments'] = False
msprefs.defaults['ignore_fields'] = []
msprefs.defaults['max_tags'] = 20
msprefs.defaults['wait_after_first_identify_result'] = 30 # seconds
msprefs.defaults['wait_after_first_cover_result'] = 60 # seconds
# Google covers are often poor quality (scans/errors) but they have high
# resolution, so they trump covers from better sources. So make sure they
# are only used if no other covers are found.
msprefs.defaults['cover_priorities'] = {'Google':2}
def create_log(ostream=None):
log = ThreadSafeLog(level=ThreadSafeLog.DEBUG)
@ -89,6 +100,39 @@ class InternalMetadataCompareKeyGen(object):
# }}}
def get_cached_cover_urls(mi):
from calibre.customize.ui import metadata_plugins
plugins = list(metadata_plugins(['identify']))
for p in plugins:
url = p.get_cached_cover_url(mi.identifiers)
if url:
yield (p, url)
def cap_author_token(token):
lt = lower(token)
if lt in ('von', 'de', 'el', 'van', 'le'):
return lt
if re.match(r'([a-z]\.){2,}$', lt) is not None:
# Normalize tokens of the form J.K. to J. K.
parts = token.split('.')
return '. '.join(map(capitalize, parts)).strip()
return capitalize(token)
def fixauthors(authors):
if not authors:
return authors
ans = []
for x in authors:
ans.append(' '.join(map(cap_author_token, x.split())))
return ans
def fixcase(x):
if x:
x = titlecase(x)
return x
class Source(Plugin):
type = _('Metadata source')
@ -104,6 +148,15 @@ class Source(Plugin):
#: during the identify phase
touched_fields = frozenset()
#: Set this to True if your plugin return HTML formatted comments
has_html_comments = False
#: Setting this to True means that the browser object will add
#: Accept-Encoding: gzip to all requests. This can speedup downloads
#: but make sure that the source actually supports gzip transfer encoding
#: correctly first
supports_gzip_transfer_encoding = False
def __init__(self, *args, **kwargs):
Plugin.__init__(self, *args, **kwargs)
self._isbn_to_identifier_cache = {}
@ -114,6 +167,13 @@ class Source(Plugin):
# Configuration {{{
def is_configured(self):
'''
Return False if your plugin needs to be configured before it can be
used. For example, it might need a username/password/API key.
'''
return True
@property
def prefs(self):
if self._config_obj is None:
@ -127,6 +187,8 @@ class Source(Plugin):
def browser(self):
if self._browser is None:
self._browser = browser(user_agent=random_user_agent())
if self.supports_gzip_transfer_encoding:
self._browser.set_handle_gzip(True)
return self._browser.clone_browser()
# }}}
@ -229,13 +291,9 @@ class Source(Plugin):
before putting the Metadata object into result_queue. You can of
course, use a custom algorithm suited to your metadata source.
'''
def fixcase(x):
if x:
x = titlecase(x)
return x
if mi.title:
mi.title = fixcase(mi.title)
mi.authors = list(map(fixcase, mi.authors))
mi.authors = fixauthors(mi.authors)
mi.tags = list(map(fixcase, mi.tags))
mi.isbn = check_isbn(mi.isbn)
@ -316,7 +374,8 @@ class Source(Plugin):
title=None, authors=None, identifiers={}, timeout=30):
'''
Download a cover and put it into result_queue. The parameters all have
the same meaning as for :meth:`identify`.
the same meaning as for :meth:`identify`. Put (self, cover_data) into
result_queue.
This method should use cached cover URLs for efficiency whenever
possible. When cached data is not present, most plugins simply call

105
src/calibre/ebooks/metadata/sources/cli.py Normal file
View File

@ -0,0 +1,105 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import sys, textwrap
from io import BytesIO
from threading import Event
from calibre import prints
from calibre.utils.config import OptionParser
from calibre.utils.magick.draw import save_cover_data_to
from calibre.ebooks.metadata import string_to_authors
from calibre.ebooks.metadata.opf2 import metadata_to_opf
from calibre.ebooks.metadata.sources.base import create_log
from calibre.ebooks.metadata.sources.identify import identify
from calibre.ebooks.metadata.sources.covers import download_cover
from calibre.utils.config import test_eight_code
def option_parser():
if not test_eight_code:
from calibre.ebooks.metadata.fetch import option_parser
return option_parser()
parser = OptionParser(textwrap.dedent(
'''\
%prog [options]
Fetch book metadata from online sources. You must specify at least one
of title, authors or ISBN.
'''
))
parser.add_option('-t', '--title', help='Book title')
parser.add_option('-a', '--authors', help='Book author(s)')
parser.add_option('-i', '--isbn', help='Book ISBN')
parser.add_option('-v', '--verbose', default=False, action='store_true',
help='Print the log to the console (stderr)')
parser.add_option('-o', '--opf', help='Output the metadata in OPF format')
parser.add_option('-c', '--cover',
help='Specify a filename. The cover, if available, will be saved to it')
parser.add_option('-d', '--timeout', default='30',
help='Timeout in seconds. Default is 30')
return parser
def main(args=sys.argv):
if not test_eight_code:
from calibre.ebooks.metadata.fetch import main
return main(args)
parser = option_parser()
opts, args = parser.parse_args(args)
buf = BytesIO()
log = create_log(buf)
abort = Event()
authors = []
if opts.authors:
authors = string_to_authors(opts.authors)
identifiers = {}
if opts.isbn:
identifiers['isbn'] = opts.isbn
results = identify(log, abort, title=opts.title, authors=authors,
identifiers=identifiers, timeout=int(opts.timeout))
if not results:
print (log, file=sys.stderr)
prints('No results found', file=sys.stderr)
raise SystemExit(1)
result = results[0]
cf = None
if opts.cover and results:
cover = download_cover(log, title=opts.title, authors=authors,
identifiers=result.identifiers, timeout=int(opts.timeout))
if cover is None:
prints('No cover found', file=sys.stderr)
else:
save_cover_data_to(cover[-1], opts.cover)
result.cover = cf = opts.cover
log = buf.getvalue()
result = (metadata_to_opf(result) if opts.opf else
unicode(result).encode('utf-8'))
if opts.verbose:
print (log, file=sys.stderr)
print (result)
if not opts.opf and opts.cover:
prints('Cover :', cf)
return 0
if __name__ == '__main__':
sys.exit(main())

178
src/calibre/ebooks/metadata/sources/covers.py Normal file
View File

@ -0,0 +1,178 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import time
from Queue import Queue, Empty
from threading import Thread, Event
from io import BytesIO
from calibre.customize.ui import metadata_plugins
from calibre.ebooks.metadata.sources.base import msprefs, create_log
from calibre.utils.magick.draw import Image, save_cover_data_to
class Worker(Thread):
def __init__(self, plugin, abort, title, authors, identifiers, timeout, rq):
Thread.__init__(self)
self.daemon = True
self.plugin = plugin
self.abort = abort
self.buf = BytesIO()
self.log = create_log(self.buf)
self.title, self.authors, self.identifiers = (title, authors,
identifiers)
self.timeout, self.rq = timeout, rq
self.time_spent = None
def run(self):
start_time = time.time()
if not self.abort.is_set():
try:
self.plugin.download_cover(self.log, self.rq, self.abort,
title=self.title, authors=self.authors,
identifiers=self.identifiers, timeout=self.timeout)
except:
self.log.exception('Failed to download cover from',
self.plugin.name)
self.time_spent = time.time() - start_time
def is_worker_alive(workers):
for w in workers:
if w.is_alive():
return True
return False
def process_result(log, result):
plugin, data = result
try:
im = Image()
im.load(data)
im.trim(10)
width, height = im.size
fmt = im.format
if width < 50 or height < 50:
raise ValueError('Image too small')
data = save_cover_data_to(im, '/cover.jpg', return_data=True)
except:
log.exception('Invalid cover from', plugin.name)
return None
return (plugin, width, height, fmt, data)
def run_download(log, results, abort,
title=None, authors=None, identifiers={}, timeout=30):
'''
Run the cover download, putting results into the queue :param:`results`.
Each result is a tuple of the form:
(plugin, width, height, fmt, bytes)
'''
plugins = [p for p in metadata_plugins(['cover']) if p.is_configured()]
rq = Queue()
workers = [Worker(p, abort, title, authors, identifiers, timeout, rq) for p
in plugins]
for w in workers:
w.start()
first_result_at = None
wait_time = msprefs['wait_after_first_cover_result']
found_results = {}
while True:
time.sleep(0.1)
try:
x = rq.get_nowait()
result = process_result(log, x)
if result is not None:
results.put(result)
found_results[result[0]] = result
if first_result_at is not None:
first_result_at = time.time()
except Empty:
pass
if not is_worker_alive(workers):
break
if first_result_at is not None and time.time() - first_result_at > wait_time:
log('Not waiting for any more results')
abort.set()
if abort.is_set():
break
while True:
try:
x = rq.get_nowait()
result = process_result(log, x)
if result is not None:
results.put(result)
found_results[result[0]] = result
except Empty:
break
for w in workers:
wlog = w.buf.getvalue().strip()
log('\n'+'*'*30, w.plugin.name, 'Covers', '*'*30)
log('Request extra headers:', w.plugin.browser.addheaders)
if w.plugin in found_results:
result = found_results[w.plugin]
log('Downloaded cover:', '%dx%d'%(result[1], result[2]))
else:
log('Failed to download valid cover')
if w.time_spent is None:
log('Download aborted')
else:
log('Took', w.time_spent, 'seconds')
if wlog:
log(wlog)
log('\n'+'*'*80)
def download_cover(log,
title=None, authors=None, identifiers={}, timeout=30):
'''
Synchronous cover download. Returns the "best" cover as per user
prefs/cover resolution.
Return cover is a tuple: (plugin, width, height, fmt, data)
Returns None if no cover is found.
'''
rq = Queue()
abort = Event()
run_download(log, rq, abort, title=title, authors=authors,
identifiers=identifiers, timeout=timeout)
results = []
while True:
try:
results.append(rq.get_nowait())
except Empty:
break
cp = msprefs['cover_priorities']
def keygen(result):
plugin, width, height, fmt, data = result
return (cp.get(plugin.name, 1), 1/(width*height))
results.sort(key=keygen)
return results[0] if results else None

24
src/calibre/ebooks/metadata/sources/google.py
View File

@ -145,21 +145,25 @@ def to_metadata(browser, log, entry_, timeout): # {{{
log.exception('Failed to parse rating')
# Cover
mi.has_google_cover = len(extra.xpath(
'//*[@rel="http://schemas.google.com/books/2008/thumbnail"]')) > 0
mi.has_google_cover = None
for x in extra.xpath(
'//*[@href and @rel="http://schemas.google.com/books/2008/thumbnail"]'):
mi.has_google_cover = x.get('href')
break
return mi
# }}}
class GoogleBooks(Source):
name = 'Google Books'
name = 'Google'
description = _('Downloads metadata from Google Books')
capabilities = frozenset(['identify', 'cover'])
touched_fields = frozenset(['title', 'authors', 'tags', 'pubdate',
'comments', 'publisher', 'identifier:isbn', 'rating',
'identifier:google']) # language currently disabled
supports_gzip_transfer_encoding = True
GOOGLE_COVER = 'http://books.google.com/books?id=%s&printsec=frontcover&img=1'
@ -212,7 +216,7 @@ class GoogleBooks(Source):
results.sort(key=self.identify_results_keygen(
title=title, authors=authors, identifiers=identifiers))
for mi in results:
cached_url = self.cover_url_from_identifiers(mi.identifiers)
cached_url = self.get_cached_cover_url(mi.identifiers)
if cached_url is not None:
break
if cached_url is None:
@ -222,9 +226,10 @@ class GoogleBooks(Source):
if abort.is_set():
return
br = self.browser
log('Downloading cover from:', cached_url)
try:
cdata = br.open_novisit(cached_url, timeout=timeout).read()
result_queue.put(cdata)
result_queue.put((self, cdata))
except:
log.exception('Failed to download cover from:', cached_url)
@ -253,9 +258,9 @@ class GoogleBooks(Source):
goog = ans.identifiers['google']
for isbn in getattr(ans, 'all_isbns', []):
self.cache_isbn_to_identifier(isbn, goog)
if ans.has_google_cover:
self.cache_identifier_to_cover_url(goog,
self.GOOGLE_COVER%goog)
if ans.has_google_cover:
self.cache_identifier_to_cover_url(goog,
self.GOOGLE_COVER%goog)
self.clean_downloaded_metadata(ans)
result_queue.put(ans)
except:
@ -270,6 +275,9 @@ class GoogleBooks(Source):
identifiers={}, timeout=30):
query = self.create_query(log, title=title, authors=authors,
identifiers=identifiers)
if not query:
log.error('Insufficient metadata to construct query')
return
br = self.browser
try:
raw = br.open_novisit(query, timeout=timeout).read()

368
src/calibre/ebooks/metadata/sources/identify.py
View File

@ -8,17 +8,21 @@ __copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import time
from datetime import datetime
from Queue import Queue, Empty
from threading import Thread
from io import BytesIO
from operator import attrgetter
from calibre.customize.ui import metadata_plugins
from calibre.ebooks.metadata.sources.base import create_log
from calibre.ebooks.metadata.sources.base import create_log, msprefs
from calibre.ebooks.metadata.xisbn import xisbn
from calibre.ebooks.metadata.book.base import Metadata
from calibre.utils.date import utc_tz
from calibre.utils.html2text import html2text
from calibre.utils.icu import lower
# How long to wait for more results after first result is found
WAIT_AFTER_FIRST_RESULT = 30 # seconds
# Download worker {{{
class Worker(Thread):
def __init__(self, plugin, kwargs, abort):
@ -31,10 +35,12 @@ class Worker(Thread):
self.log = create_log(self.buf)
def run(self):
start = time.time()
try:
self.plugin.identify(self.log, self.rq, self.abort, **self.kwargs)
except:
self.log.exception('Plugin', self.plugin.name, 'failed')
self.plugin.dl_time_spent = time.time() - start
def is_worker_alive(workers):
for w in workers:
@ -42,28 +48,235 @@ def is_worker_alive(workers):
return True
return False
def identify(log, abort, title=None, authors=None, identifiers=[], timeout=30):
# }}}
# Merge results from different sources {{{
class ISBNMerge(object):
def __init__(self):
self.pools = {}
self.isbnless_results = []
def isbn_in_pool(self, isbn):
if isbn:
for isbns, pool in self.pools.iteritems():
if isbn in isbns:
return pool
return None
def pool_has_result_from_same_source(self, pool, result):
results = pool[1]
for r in results:
if r.identify_plugin is result.identify_plugin:
return True
return False
def add_result(self, result):
isbn = result.isbn
if isbn:
pool = self.isbn_in_pool(isbn)
if pool is None:
isbns, min_year = xisbn.get_isbn_pool(isbn)
if not isbns:
isbns = frozenset([isbn])
self.pools[isbns] = pool = (min_year, [])
if not self.pool_has_result_from_same_source(pool, result):
pool[1].append(result)
else:
self.isbnless_results.append(result)
def finalize(self):
has_isbn_result = False
for results in self.pools.itervalues():
if results:
has_isbn_result = True
break
self.has_isbn_result = has_isbn_result
if has_isbn_result:
self.merge_isbn_results()
else:
results = sorted(self.isbnless_results,
key=attrgetter('relevance_in_source'))
# Pick only the most relevant result from each source
self.results = []
seen = set()
for result in results:
if result.identify_plugin not in seen:
seen.add(result.identify_plugin)
self.results.append(result)
result.average_source_relevance = \
result.relevance_in_source
self.merge_metadata_results()
return self.results
def merge_metadata_results(self):
' Merge results with identical title and authors '
groups = {}
for result in self.results:
title = lower(result.title if result.title else '')
key = (title, tuple([lower(x) for x in result.authors]))
if key not in groups:
groups[key] = []
groups[key].append(result)
if len(groups) != len(self.results):
self.results = []
for rgroup in groups.itervalues():
rel = [r.average_source_relevance for r in rgroup]
if len(rgroup) > 1:
result = self.merge(rgroup, None, do_asr=False)
result.average_source_relevance = sum(rel)/len(rel)
else:
result = rgroup[0]
self.results.append(result)
self.results.sort(key=attrgetter('average_source_relevance'))
def merge_isbn_results(self):
self.results = []
for min_year, results in self.pools.itervalues():
if results:
self.results.append(self.merge(results, min_year))
self.results.sort(key=attrgetter('average_source_relevance'))
def length_merge(self, attr, results, null_value=None, shortest=True):
values = [getattr(x, attr) for x in results if not x.is_null(attr)]
values = [x for x in values if len(x) > 0]
if not values:
return null_value
values.sort(key=len, reverse=not shortest)
return values[0]
def random_merge(self, attr, results, null_value=None):
values = [getattr(x, attr) for x in results if not x.is_null(attr)]
return values[0] if values else null_value
def merge(self, results, min_year, do_asr=True):
ans = Metadata(_('Unknown'))
# We assume the shortest title has the least cruft in it
ans.title = self.length_merge('title', results, null_value=ans.title)
# No harm in having extra authors, maybe something useful like an
# editor or translator
ans.authors = self.length_merge('authors', results,
null_value=ans.authors, shortest=False)
# We assume the shortest publisher has the least cruft in it
ans.publisher = self.length_merge('publisher', results,
null_value=ans.publisher)
# We assume the smallest set of tags has the least cruft in it
ans.tags = self.length_merge('tags', results,
null_value=ans.tags)
# We assume the longest series has the most info in it
ans.series = self.length_merge('series', results,
null_value=ans.series, shortest=False)
for r in results:
if r.series and r.series == ans.series:
ans.series_index = r.series_index
break
# Average the rating over all sources
ratings = []
for r in results:
rating = r.rating
if rating and rating > 0 and rating <= 5:
ratings.append(rating)
if ratings:
ans.rating = sum(ratings)/len(ratings)
# Smallest language is likely to be valid
ans.language = self.length_merge('language', results,
null_value=ans.language)
# Choose longest comments
ans.comments = self.length_merge('comments', results,
null_value=ans.comments, shortest=False)
# Published date
if min_year:
min_date = datetime(min_year, 1, 2, tzinfo=utc_tz)
ans.pubdate = min_date
else:
min_date = datetime(3001, 1, 1, tzinfo=utc_tz)
for r in results:
if r.pubdate is not None and r.pubdate < min_date:
min_date = r.pubdate
if min_date.year < 3000:
ans.pubdate = min_date
# Identifiers
for r in results:
ans.identifiers.update(r.identifiers)
# Cover URL
ans.has_cached_cover_url = bool([r for r in results if
getattr(r, 'has_cached_cover_url', False)])
# Merge any other fields with no special handling (random merge)
touched_fields = set()
for r in results:
if hasattr(r, 'identify_plugin'):
touched_fields |= r.identify_plugin.touched_fields
for f in touched_fields:
if f.startswith('identifier:') or not ans.is_null(f):
continue
setattr(ans, f, self.random_merge(f, results,
null_value=getattr(ans, f)))
if do_asr:
avg = [x.relevance_in_source for x in results]
avg = sum(avg)/len(avg)
ans.average_source_relevance = avg
return ans
def merge_identify_results(result_map, log):
isbn_merge = ISBNMerge()
for plugin, results in result_map.iteritems():
for result in results:
isbn_merge.add_result(result)
return isbn_merge.finalize()
# }}}
def identify(log, abort, # {{{
title=None, authors=None, identifiers={}, timeout=30):
start_time = time.time()
plugins = list(metadata_plugins['identify'])
plugins = [p for p in metadata_plugins(['identify']) if p.is_configured()]
kwargs = {
'title': title,
'authors': authors,
'identifiers': identifiers,
'timeout': timeout,
'title': title,
'authors': authors,
'identifiers': identifiers,
'timeout': timeout,
}
log('Running identify query with parameters:')
log(kwargs)
log('Using plugins:', ', '.join([p.name for p in plugins]))
log('The log (if any) from individual plugins is below')
log('The log from individual plugins is below')
workers = [Worker(p, kwargs, abort) for p in plugins]
for w in workers:
w.start()
first_result_at = None
results = dict.fromkeys(plugins, [])
results = {}
for p in plugins:
results[p] = []
logs = dict([(w.plugin, w.buf) for w in workers])
def get_results():
found = False
@ -77,6 +290,7 @@ def identify(log, abort, title=None, authors=None, identifiers=[], timeout=30):
found = True
return found
wait_time = msprefs['wait_after_first_identify_result']
while True:
time.sleep(0.2)
@ -86,76 +300,118 @@ def identify(log, abort, title=None, authors=None, identifiers=[], timeout=30):
if not is_worker_alive(workers):
break
if (first_result_at is not None and time.time() - first_result_at <
WAIT_AFTER_FIRST_RESULT):
if (first_result_at is not None and time.time() - first_result_at >
wait_time):
log('Not waiting any longer for more results')
abort.set()
break
get_results()
while not abort.is_set() and get_results():
pass
sort_kwargs = dict(kwargs)
for k in list(sort_kwargs.iterkeys()):
if k not in ('title', 'authors', 'identifiers'):
sort_kwargs.pop(k)
for plugin, results in results.iteritems():
results.sort(key=plugin.identify_results_keygen(**sort_kwargs))
plog = plugin.buf.getvalue().strip()
longest, lp = -1, ''
for plugin, presults in results.iteritems():
presults.sort(key=plugin.identify_results_keygen(**sort_kwargs))
plog = logs[plugin].getvalue().strip()
log('\n'+'*'*30, plugin.name, '*'*30)
log('Request extra headers:', plugin.browser.addheaders)
log('Found %d results'%len(presults))
time_spent = getattr(plugin, 'dl_time_spent', None)
if time_spent is None:
log('Downloading was aborted')
longest, lp = -1, plugin.name
else:
log('Downloading from', plugin.name, 'took', time_spent)
if time_spent > longest:
longest, lp = time_spent, plugin.name
for r in presults:
log('\n\n---')
log(unicode(r))
if plog:
log('\n'+'*'*35, plugin.name, '*'*35)
log('Found %d results'%len(results))
log(plog)
log('\n'+'*'*80)
log('\n'+'*'*80)
for i, result in enumerate(results):
for i, result in enumerate(presults):
result.relevance_in_source = i
result.has_cached_cover_url = \
plugin.get_cached_cover_url(result.identifiers) is not None
result.identify_plugin = plugin
log('The identify phase took %.2f seconds'%(time.time() - start_time))
log('The longest time (%f) was taken by:'%longest, lp)
log('Merging results from different sources and finding earliest',
'publication dates')
start_time = time.time()
merged_results = merge_identify_results(results, log)
results = merge_identify_results(results, log)
log('We have %d merged results, merging took: %.2f seconds' %
(len(merged_results), time.time() - start_time))
(len(results), time.time() - start_time))
class ISBNMerge(object):
def __init__(self):
self.pools = {}
def isbn_in_pool(self, isbn):
if isbn:
for p in self.pools:
if isbn in p:
return p
return None
def pool_has_result_from_same_source(self, pool, result):
results = self.pools[pool][1]
if msprefs['txt_comments']:
for r in results:
if r.identify_plugin is result.identify_plugin:
return True
return False
if r.plugin.has_html_comments and r.comments:
r.comments = html2text(r.comments)
def add_result(self, result, isbn):
pool = self.isbn_in_pool(isbn)
if pool is None:
isbns, min_year = xisbn.get_isbn_pool(isbn)
if not isbns:
isbns = frozenset([isbn])
self.pool[isbns] = pool = (min_year, [])
dummy = Metadata(_('Unknown'))
max_tags = msprefs['max_tags']
for r in results:
for f in msprefs['ignore_fields']:
setattr(r, f, getattr(dummy, f))
r.tags = r.tags[:max_tags]
if not self.pool_has_result_from_same_source(pool, result):
pool[1].append(result)
return results
# }}}
def merge_identify_results(result_map, log):
for plugin, results in result_map.iteritems():
for result in results:
isbn = result.isbn
if isbn:
isbns, min_year = xisbn.get_isbn_pool(isbn)
if __name__ == '__main__': # tests {{{
# To run these test use: calibre-debug -e
# src/calibre/ebooks/metadata/sources/identify.py
from calibre.ebooks.metadata.sources.test import (test_identify,
title_test, authors_test)
tests = [
( # An e-book ISBN not on Amazon, one of the authors is
# unknown to Amazon
{'identifiers':{'isbn': '9780307459671'},
'title':'Invisible Gorilla', 'authors':['Christopher Chabris']},
[title_test('The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us',
exact=True), authors_test(['Christopher Chabris', 'Daniel Simons'])]
),
( # Test absence of identifiers
{'title':'Learning Python',
'authors':['Lutz']},
[title_test('Learning Python',
exact=True), authors_test(['Mark Lutz'])
]
),
( # Sophisticated comment formatting
{'identifiers':{'isbn': '9781416580829'}},
[title_test('Angels & Demons',
exact=True), authors_test(['Dan Brown'])]
),
( # No ISBN
{'title':'Justine', 'authors':['Durrel']},
[title_test('Justine', exact=True),
authors_test(['Lawrence Durrel'])]
),
( # A newer book
{'identifiers':{'isbn': '9780316044981'}},
[title_test('The Heroes', exact=True),
authors_test(['Joe Abercrombie'])]
),
]
#test_identify(tests[1:2])
test_identify(tests)
# }}}

43
src/calibre/ebooks/metadata/sources/isbndb.py Normal file
View File

@ -0,0 +1,43 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
from calibre.ebooks.metadata.sources.base import Source
class ISBNDB(Source):
name = 'ISBNDB'
description = _('Downloads metadata from isbndb.com')
capabilities = frozenset(['identify'])
touched_fields = frozenset(['title', 'authors',
'identifier:isbn', 'comments', 'publisher'])
supports_gzip_transfer_encoding = True
def __init__(self, *args, **kwargs):
Source.__init__(self, *args, **kwargs)
prefs = self.prefs
prefs.defaults['key_migrated'] = False
prefs.defaults['isbndb_key'] = None
if not prefs['key_migrated']:
prefs['key_migrated'] = True
try:
from calibre.customize.ui import config
key = config['plugin_customization']['IsbnDB']
prefs['isbndb_key'] = key
except:
pass
self.isbndb_key = prefs['isbndb_key']
def is_configured(self):
return self.isbndb_key is not None

2
src/calibre/ebooks/metadata/sources/openlibrary.py
View File

@ -26,7 +26,7 @@ class OpenLibrary(Source):
br = self.browser
try:
ans = br.open_novisit(self.OPENLIBRARY%isbn, timeout=timeout).read()
result_queue.put(ans)
result_queue.put((self, ans))
except Exception as e:
if callable(getattr(e, 'getcode', None)) and e.getcode() == 404:
log.error('No cover for ISBN: %r found'%isbn)

84
src/calibre/ebooks/metadata/sources/test.py
View File

@ -14,7 +14,8 @@ from threading import Event
from calibre.customize.ui import metadata_plugins
from calibre import prints, sanitize_file_name2
from calibre.ebooks.metadata import check_isbn
from calibre.ebooks.metadata.sources.base import create_log
from calibre.ebooks.metadata.sources.base import (create_log,
get_cached_cover_urls)
def isbn_test(isbn):
isbn_ = check_isbn(isbn)
@ -45,8 +46,80 @@ def authors_test(authors):
return test
def init_test(tdir_name):
tdir = tempfile.gettempdir()
lf = os.path.join(tdir, tdir_name.replace(' ', '')+'_identify_test.txt')
log = create_log(open(lf, 'wb'))
abort = Event()
return tdir, lf, log, abort
def test_identify_plugin(name, tests):
def test_identify(tests): # {{{
'''
:param tests: List of 2-tuples. Each two tuple is of the form (args,
test_funcs). args is a dict of keyword arguments to pass to
the identify method. test_funcs are callables that accept a
Metadata object and return True iff the object passes the
test.
'''
from calibre.ebooks.metadata.sources.identify import identify
tdir, lf, log, abort = init_test('Full Identify')
prints('Log saved to', lf)
times = []
for kwargs, test_funcs in tests:
log('#'*80)
log('### Running test with:', kwargs)
log('#'*80)
prints('Running test with:', kwargs)
args = (log, abort)
start_time = time.time()
results = identify(*args, **kwargs)
total_time = time.time() - start_time
times.append(total_time)
if not results:
prints('identify failed to find any results')
break
prints('Found', len(results), 'matches:', end=' ')
prints('Smaller relevance means better match')
for i, mi in enumerate(results):
prints('*'*30, 'Relevance:', i, '*'*30)
prints(mi)
prints('\nCached cover URLs :',
[x[0].name for x in get_cached_cover_urls(mi)])
prints('*'*75, '\n\n')
possibles = []
for mi in results:
test_failed = False
for tfunc in test_funcs:
if not tfunc(mi):
test_failed = True
break
if not test_failed:
possibles.append(mi)
if not possibles:
prints('ERROR: No results that passed all tests were found')
prints('Log saved to', lf)
raise SystemExit(1)
if results[0] is not possibles[0]:
prints('Most relevant result failed the tests')
raise SystemExit(1)
log('\n\n')
prints('Average time per query', sum(times)/len(times))
prints('Full log is at:', lf)
# }}}
def test_identify_plugin(name, tests): # {{{
'''
:param name: Plugin name
:param tests: List of 2-tuples. Each two tuple is of the form (args,
@ -61,11 +134,9 @@ def test_identify_plugin(name, tests):
plugin = x
break
prints('Testing the identify function of', plugin.name)
prints('Using extra headers:', plugin.browser.addheaders)
tdir = tempfile.gettempdir()
lf = os.path.join(tdir, plugin.name.replace(' ', '')+'_identify_test.txt')
log = create_log(open(lf, 'wb'))
abort = Event()
tdir, lf, log, abort = init_test(plugin.name)
prints('Log saved to', lf)
times = []
@ -159,4 +230,5 @@ def test_identify_plugin(name, tests):
if os.stat(lf).st_size > 10:
prints('There were some errors/warnings, see log', lf)
# }}}

6
src/calibre/ebooks/metadata/xisbn.py
View File

@ -73,7 +73,11 @@ class xISBN(object):
def get_isbn_pool(self, isbn):
data = self.get_data(isbn)
isbns = frozenset([x.get('isbn') for x in data if 'isbn' in x])
raw = tuple(x.get('isbn') for x in data if 'isbn' in x)
isbns = []
for x in raw:
isbns += x
isbns = frozenset(isbns)
min_year = 100000
for x in data:
try:

6
src/calibre/ebooks/mobi/writer.py
View File

@ -282,8 +282,8 @@ class Serializer(object):
buffer.write('="')
self.serialize_text(val, quot=True)
buffer.write('"')
buffer.write('>')
if elem.text or len(elem) > 0:
buffer.write('>')
if elem.text:
self.anchor_offset = None
self.serialize_text(elem.text)
@ -292,9 +292,7 @@ class Serializer(object):
if child.tail:
self.anchor_offset = None
self.serialize_text(child.tail)
buffer.write('</%s>' % tag)
else:
buffer.write('/>')
buffer.write('</%s>' % tag)
def serialize_text(self, text, quot=False):
text = text.replace('&', '&amp;')

29
src/calibre/ebooks/oeb/stylizer.py
View File

@ -17,6 +17,8 @@ from cssutils.css import CSSStyleRule, CSSPageRule, CSSStyleDeclaration, \
from cssutils import profile as cssprofiles
from lxml import etree
from lxml.cssselect import css_to_xpath, ExpressionError, SelectorSyntaxError
from calibre import force_unicode
from calibre.ebooks.oeb.base import XHTML, XHTML_NS, CSS_MIME, OEB_STYLES
from calibre.ebooks.oeb.base import XPNSMAP, xpath, urlnormalize
from calibre.ebooks.oeb.profile import PROFILES
@ -95,6 +97,10 @@ class CSSSelector(etree.XPath):
def __init__(self, css, namespaces=XPNSMAP):
css = self.MIN_SPACE_RE.sub(r'\1', css)
if isinstance(css, unicode):
# Workaround for bug in lxml on windows/OS X that causes a massive
# memory leak with non ASCII selectors
css = css.encode('ascii', 'ignore').decode('ascii')
try:
path = css_to_xpath(css)
except UnicodeEncodeError: # Bug in css_to_xpath
@ -140,13 +146,22 @@ class Stylizer(object):
log=logging.getLogger('calibre.css'))
self.font_face_rules = []
for elem in head:
if elem.tag == XHTML('style') and elem.text \
and elem.get('type', CSS_MIME) in OEB_STYLES:
text = XHTML_CSS_NAMESPACE + elem.text
text = oeb.css_preprocessor(text)
stylesheet = parser.parseString(text, href=cssname)
stylesheet.namespaces['h'] = XHTML_NS
stylesheets.append(stylesheet)
if (elem.tag == XHTML('style') and
elem.get('type', CSS_MIME) in OEB_STYLES):
text = elem.text if elem.text else u''
for x in elem:
t = getattr(x, 'text', None)
if t:
text += u'\n\n' + force_unicode(t, u'utf-8')
t = getattr(x, 'tail', None)
if t:
text += u'\n\n' + force_unicode(t, u'utf-8')
if text:
text = XHTML_CSS_NAMESPACE + elem.text
text = oeb.css_preprocessor(text)
stylesheet = parser.parseString(text, href=cssname)
stylesheet.namespaces['h'] = XHTML_NS
stylesheets.append(stylesheet)
elif elem.tag == XHTML('link') and elem.get('href') \
and elem.get('rel', 'stylesheet').lower() == 'stylesheet' \
and elem.get('type', CSS_MIME).lower() in OEB_STYLES:

5
src/calibre/ebooks/oeb/transforms/page_margin.py
View File

@ -20,7 +20,8 @@ class RemoveAdobeMargins(object):
self.oeb, self.opts, self.log = oeb, opts, log
for item in self.oeb.manifest:
if item.media_type == 'application/vnd.adobe-page-template+xml':
if item.media_type in ('application/vnd.adobe-page-template+xml',
'application/vnd.adobe.page-template+xml'):
self.log('Removing page margins specified in the'
' Adobe page template')
for elem in item.data.xpath(
@ -35,7 +36,7 @@ class RemoveFakeMargins(object):
'''
Remove left and right margins from paragraph/divs if the same margin is specified
on almost all the elements of at that level.
on almost all the elements at that level.
Must be called only after CSS flattening
'''

8
src/calibre/ebooks/pdf/fonts.cpp
View File

@ -72,6 +72,7 @@ XMLFont::XMLFont(string* font_name, double size, GfxRGB rgb) :
size(size-1), line_size(-1.0), italic(false), bold(false), font_name(font_name),
font_family(NULL), color(rgb) {
if (!this->font_name) this->font_name = new string(DEFAULT_FONT_FAMILY);
this->font_family = family_name(this->font_name);
if (strcasestr(font_name->c_str(), "bold")) this->bold = true;
@ -134,7 +135,12 @@ Fonts::size_type Fonts::add_font(XMLFont *f) {
}
Fonts::size_type Fonts::add_font(string* font_name, double size, GfxRGB rgb) {
XMLFont *f = new XMLFont(font_name, size, rgb);
XMLFont *f = NULL;
if (font_name == NULL)
font_name = new string("Unknown");
// font_name must not be deleted
f = new XMLFont(font_name, size, rgb);
return this->add_font(f);
}

2
src/calibre/ebooks/txt/markdownml.py
View File

@ -37,7 +37,7 @@ class MarkdownMLizer(object):
if not self.opts.keep_links:
html = re.sub(r'<\s*/*\s*a[^>]*>', '', html)
if not self.opts.keep_image_references:
html = re.sub(r'<\s*img[^>]*>', '', html)\
html = re.sub(r'<\s*img[^>]*>', '', html)
text = html2text(html)

2
src/calibre/ebooks/txt/output.py
View File

@ -11,7 +11,7 @@ from lxml import etree
from calibre.customize.conversion import OutputFormatPlugin, \
OptionRecommendation
from calibre.ebooks.oeb.base import OEB_IMAGES
from calibre.ebooks.oeb.base import OEB_IMAGES
from calibre.ebooks.txt.txtml import TXTMLizer
from calibre.ebooks.txt.newlines import TxtNewlines, specified_newlines
from calibre.ptempfile import TemporaryDirectory, TemporaryFile

3
src/calibre/gui2/actions/__init__.py
View File

@ -145,11 +145,10 @@ class InterfaceAction(QObject):
ans[candidate] = zf.read(candidate)
return ans
def genesis(self):
'''
Setup this plugin. Only called once during initialization. self.gui is
available. The action secified by :attr:`action_spec` is available as
available. The action specified by :attr:`action_spec` is available as
``self.qaction``.
'''
pass

8
src/calibre/gui2/actions/catalog.py
View File

@ -31,10 +31,10 @@ class GenerateCatalogAction(InterfaceAction):
_('No books selected for catalog generation'),
show=True)
db = self.gui.library_view.model().db
dbspec = {}
for id in ids:
dbspec[id] = {'ondevice': db.ondevice(id, index_is_id=True)}
db = self.gui.library_view.model().db
dbspec = {}
for id in ids:
dbspec[id] = {'ondevice': db.ondevice(id, index_is_id=True)}
# Calling gui2.tools:generate_catalog()
ret = generate_catalog(self.gui, dbspec, ids, self.gui.device_manager,

7
src/calibre/gui2/actions/choose_library.py
View File

@ -8,7 +8,7 @@ __docformat__ = 'restructuredtext en'
import os, shutil
from functools import partial
from PyQt4.Qt import QMenu, Qt, QInputDialog
from PyQt4.Qt import QMenu, Qt, QInputDialog, QToolButton
from calibre import isbytestring
from calibre.constants import filesystem_encoding
@ -88,6 +88,9 @@ class ChooseLibraryAction(InterfaceAction):
type=Qt.QueuedConnection)
self.stats = LibraryUsageStats()
self.popup_type = (QToolButton.InstantPopup if len(self.stats.stats) > 1 else
QToolButton.MenuButtonPopup)
self.create_action(spec=(_('Switch/create library...'), 'lt.png', None,
None), attr='action_choose')
self.action_choose.triggered.connect(self.choose_library,
@ -123,6 +126,7 @@ class ChooseLibraryAction(InterfaceAction):
type=Qt.QueuedConnection)
self.choose_menu.addAction(ac)
self.rename_separator = self.choose_menu.addSeparator()
self.maintenance_menu = QMenu(_('Library Maintenance'))
@ -172,6 +176,7 @@ class ChooseLibraryAction(InterfaceAction):
return
db = self.gui.library_view.model().db
locations = list(self.stats.locations(db))
for ac in self.switch_actions:
ac.setVisible(False)
self.quick_menu.clear()

54
src/calibre/gui2/actions/edit_metadata.py
View File

@ -17,6 +17,7 @@ from calibre.gui2.dialogs.confirm_delete import confirm
from calibre.gui2.dialogs.tag_list_editor import TagListEditor
from calibre.gui2.actions import InterfaceAction
from calibre.utils.icu import sort_key
from calibre.utils.config import test_eight_code
class EditMetadataAction(InterfaceAction):
@ -133,8 +134,6 @@ class EditMetadataAction(InterfaceAction):
row_list = [r.row() for r in rows]
current_row = 0
changed = set([])
db = self.gui.library_view.model().db
if len(row_list) == 1:
cr = row_list[0]
@ -142,6 +141,27 @@ class EditMetadataAction(InterfaceAction):
list(range(self.gui.library_view.model().rowCount(QModelIndex())))
current_row = row_list.index(cr)
func = (self.do_edit_metadata if test_eight_code else
self.do_edit_metadata_old)
changed, rows_to_refresh = func(row_list, current_row)
m = self.gui.library_view.model()
if rows_to_refresh:
m.refresh_rows(rows_to_refresh)
if changed:
m.refresh_ids(list(changed))
current = self.gui.library_view.currentIndex()
if self.gui.cover_flow:
self.gui.cover_flow.dataChanged()
m.current_changed(current, previous)
self.gui.tags_view.recount()
def do_edit_metadata_old(self, row_list, current_row):
changed = set([])
db = self.gui.library_view.model().db
while True:
prev = next_ = None
if current_row > 0:
@ -166,16 +186,30 @@ class EditMetadataAction(InterfaceAction):
current_row += d.row_delta
self.gui.library_view.set_current_row(current_row)
self.gui.library_view.scroll_to_row(current_row)
return changed, set()
def do_edit_metadata(self, row_list, current_row):
from calibre.gui2.metadata.single import edit_metadata
db = self.gui.library_view.model().db
changed, rows_to_refresh = edit_metadata(db, row_list, current_row,
parent=self.gui, view_slot=self.view_format_callback,
set_current_callback=self.set_current_callback)
return changed, rows_to_refresh
def set_current_callback(self, id_):
db = self.gui.library_view.model().db
current_row = db.row(id_)
self.gui.library_view.set_current_row(current_row)
self.gui.library_view.scroll_to_row(current_row)
def view_format_callback(self, id_, fmt):
view = self.gui.iactions['View']
if id_ is None:
view._view_file(fmt)
else:
db = self.gui.library_view.model().db
view.view_format(db.row(id_), fmt)
if changed:
self.gui.library_view.model().refresh_ids(list(changed))
current = self.gui.library_view.currentIndex()
m = self.gui.library_view.model()
if self.gui.cover_flow:
self.gui.cover_flow.dataChanged()
m.current_changed(current, previous)
self.gui.tags_view.recount()
def edit_bulk_metadata(self, checked):
'''

7
src/calibre/gui2/book_details.py
View File

@ -270,6 +270,8 @@ class BookInfo(QWebView):
<style type="text/css">
body, td {background-color: transparent; font-size: %dpx; color: %s }
a { text-decoration: none; color: blue }
div.description { margin-top: 0; padding-top: 0; text-indent: 0 }
table { margin-bottom: 0; padding-bottom: 0; }
</style>
</head>
<body>
@ -278,9 +280,10 @@ class BookInfo(QWebView):
<html>
'''%(f, c)
if self.vertical:
extra = ''
if comments:
rows += u'<tr><td colspan="2">%s</td></tr>'%comments
self.setHtml(templ%(u'<table>%s</table>'%rows))
extra = u'<div class="description">%s</div>'%comments
self.setHtml(templ%(u'<table>%s</table>%s'%(rows, extra)))
else:
left_pane = u'<table>%s</table>'%rows
right_pane = u'<div>%s</div>'%comments

26
src/calibre/gui2/convert/htmlz_output.py Normal file
View File

@ -0,0 +1,26 @@
# -*- coding: utf-8 -*-
__license__ = 'GPL 3'
__copyright__ = '2009, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
from calibre.gui2.convert.htmlz_output_ui import Ui_Form
from calibre.gui2.convert import Widget
format_model = None
class PluginWidget(Widget, Ui_Form):
TITLE = _('HTMLZ Output')
HELP = _('Options specific to')+' HTMLZ '+_('output')
COMMIT_NAME = 'htmlz_output'
ICON = I('mimetypes/html.png')
def __init__(self, parent, get_option, get_help, db=None, book_id=None):
Widget.__init__(self, parent, ['htmlz_css_type', 'htmlz_class_style'])
self.db, self.book_id = db, book_id
for x in get_option('htmlz_css_type').option.choices:
self.opt_htmlz_css_type.addItem(x)
for x in get_option('htmlz_class_style').option.choices:
self.opt_htmlz_class_style.addItem(x)
self.initialize_options(get_option, get_help, db, book_id)

61
src/calibre/gui2/convert/htmlz_output.ui Normal file
View File

@ -0,0 +1,61 @@
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>Form</class>
<widget class="QWidget" name="Form">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>438</width>
<height>300</height>
</rect>
</property>
<property name="windowTitle">
<string>Form</string>
</property>
<layout class="QGridLayout" name="gridLayout">
<item row="2" column="0">
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>246</height>
</size>
</property>
</spacer>
</item>
<item row="0" column="0">
<widget class="QLabel" name="label">
<property name="text">
<string>How to handle CSS</string>
</property>
<property name="buddy">
<cstring>opt_htmlz_css_type</cstring>
</property>
</widget>
</item>
<item row="0" column="1">
<widget class="QComboBox" name="opt_htmlz_css_type">
<property name="minimumContentsLength">
<number>20</number>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_2">
<property name="text">
<string>How to handle class based CSS</string>
</property>
</widget>
</item>
<item row="1" column="1">
<widget class="QComboBox" name="opt_htmlz_class_style"/>
</item>
</layout>
</widget>
<resources/>
<connections/>
</ui>

14
src/calibre/gui2/custom_column_widgets.py
View File

@ -62,7 +62,7 @@ class Bool(Base):
w = self.widgets[1]
items = [_('Yes'), _('No'), _('Undefined')]
icons = [I('ok.png'), I('list_remove.png'), I('blank.png')]
if tweaks['bool_custom_columns_are_tristate'] == 'no':
if not self.db.prefs.get('bools_are_tristate'):
items = items[:-1]
icons = icons[:-1]
for icon, text in zip(icons, items):
@ -70,7 +70,7 @@ class Bool(Base):
def setter(self, val):
val = {None: 2, False: 1, True: 0}[val]
if tweaks['bool_custom_columns_are_tristate'] == 'no' and val == 2:
if not self.db.prefs.get('bools_are_tristate') and val == 2:
val = 1
self.widgets[1].setCurrentIndex(val)
@ -549,7 +549,7 @@ class BulkBool(BulkBase, Bool):
value = None
for book_id in book_ids:
val = self.db.get_custom(book_id, num=self.col_id, index_is_id=True)
if tweaks['bool_custom_columns_are_tristate'] == 'no' and val is None:
if not self.db.prefs.get('bools_are_tristate') and val is None:
val = False
if value is not None and value != val:
return None
@ -559,7 +559,7 @@ class BulkBool(BulkBase, Bool):
def setup_ui(self, parent):
self.make_widgets(parent, QComboBox)
items = [_('Yes'), _('No')]
if tweaks['bool_custom_columns_are_tristate'] == 'no':
if not self.db.prefs.get('bools_are_tristate'):
items.append('')
else:
items.append(_('Undefined'))
@ -571,7 +571,7 @@ class BulkBool(BulkBase, Bool):
def getter(self):
val = self.main_widget.currentIndex()
if tweaks['bool_custom_columns_are_tristate'] == 'no':
if not self.db.prefs.get('bools_are_tristate'):
return {2: False, 1: False, 0: True}[val]
else:
return {2: None, 1: False, 0: True}[val]
@ -586,13 +586,13 @@ class BulkBool(BulkBase, Bool):
return
val = self.gui_val
val = self.normalize_ui_val(val)
if tweaks['bool_custom_columns_are_tristate'] == 'no' and val is None:
if not self.db.prefs.get('bools_are_tristate') and val is None:
val = False
self.db.set_custom_bulk(book_ids, val, num=self.col_id, notify=notify)
def a_c_checkbox_changed(self):
if not self.ignore_change_signals:
if tweaks['bool_custom_columns_are_tristate'] == 'no' and \
if not self.db.prefs.get('bools_are_tristate') and \
self.main_widget.currentIndex() == 2:
self.a_c_checkbox.setChecked(False)
else:

162
src/calibre/gui2/dialogs/book_info.ui
View File

@ -7,15 +7,25 @@
<x>0</x>
<y>0</y>
<width>917</width>
<height>480</height>
<height>492</height>
</rect>
</property>
<property name="windowTitle">
<string>Dialog</string>
</property>
<property name="windowIcon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/metadata.png</normaloff>:/images/metadata.png</iconset>
</property>
<layout class="QGridLayout" name="gridLayout">
<item row="0" column="0" colspan="2">
<widget class="QLabel" name="title">
<property name="font">
<font>
<weight>75</weight>
<bold>true</bold>
</font>
</property>
<property name="text">
<string>TextLabel</string>
</property>
@ -24,86 +34,104 @@
</property>
</widget>
</item>
<item row="1" column="0">
<item row="1" column="0" rowspan="3">
<widget class="CoverView" name="cover"/>
</item>
<item row="1" column="1">
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QLabel" name="text">
<property name="text">
<string>TextLabel</string>
</property>
<property name="alignment">
<set>Qt::AlignLeading|Qt::AlignLeft|Qt::AlignTop</set>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<widget class="QGroupBox" name="groupBox">
<property name="title">
<string>Comments</string>
</property>
<layout class="QVBoxLayout" name="verticalLayout_2">
<item>
<widget class="QWebView" name="comments">
<property name="sizePolicy">
<sizepolicy hsizetype="Preferred" vsizetype="Expanding">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="maximumSize">
<size>
<width>350</width>
<height>16777215</height>
</size>
</property>
<property name="url">
<url>
<string>about:blank</string>
</url>
</property>
</widget>
</item>
</layout>
</widget>
</item>
<item>
<widget class="QCheckBox" name="fit_cover">
<property name="text">
<string>Fit &amp;cover within view</string>
</property>
</widget>
</item>
<item>
<layout class="QHBoxLayout" name="horizontalLayout">
<widget class="QScrollArea" name="scrollArea">
<property name="frameShape">
<enum>QFrame::NoFrame</enum>
</property>
<property name="widgetResizable">
<bool>true</bool>
</property>
<widget class="QWidget" name="scrollAreaWidgetContents">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>435</width>
<height>670</height>
</rect>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QPushButton" name="previous_button">
<widget class="QLabel" name="text">
<property name="text">
<string>&amp;Previous</string>
<string>TextLabel</string>
</property>
<property name="icon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/previous.png</normaloff>:/images/previous.png</iconset>
<property name="alignment">
<set>Qt::AlignLeading|Qt::AlignLeft|Qt::AlignTop</set>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="next_button">
<property name="text">
<string>&amp;Next</string>
</property>
<property name="icon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/next.png</normaloff>:/images/next.png</iconset>
<widget class="QGroupBox" name="groupBox">
<property name="title">
<string>Comments</string>
</property>
<layout class="QVBoxLayout" name="verticalLayout_2">
<item>
<widget class="QWebView" name="comments">
<property name="sizePolicy">
<sizepolicy hsizetype="Preferred" vsizetype="Expanding">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="maximumSize">
<size>
<width>350</width>
<height>16777215</height>
</size>
</property>
<property name="url">
<url>
<string>about:blank</string>
</url>
</property>
</widget>
</item>
</layout>
</widget>
</item>
</layout>
</widget>
</widget>
</item>
<item row="2" column="1">
<widget class="QCheckBox" name="fit_cover">
<property name="text">
<string>Fit &amp;cover within view</string>
</property>
</widget>
</item>
<item row="3" column="1">
<layout class="QHBoxLayout" name="horizontalLayout">
<item>
<widget class="QPushButton" name="previous_button">
<property name="text">
<string>&amp;Previous</string>
</property>
<property name="icon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/previous.png</normaloff>:/images/previous.png</iconset>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="next_button">
<property name="text">
<string>&amp;Next</string>
</property>
<property name="icon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/next.png</normaloff>:/images/next.png</iconset>
</property>
</widget>
</item>
</layout>
</item>

10
src/calibre/gui2/layout.py
View File

@ -7,9 +7,9 @@ __docformat__ = 'restructuredtext en'
from functools import partial
from PyQt4.Qt import QIcon, Qt, QWidget, QToolBar, QSize, \
pyqtSignal, QToolButton, QMenu, \
QObject, QVBoxLayout, QSizePolicy, QLabel, QHBoxLayout, QActionGroup
from PyQt4.Qt import (QIcon, Qt, QWidget, QToolBar, QSize,
pyqtSignal, QToolButton, QMenu,
QObject, QVBoxLayout, QSizePolicy, QLabel, QHBoxLayout, QActionGroup)
from calibre.constants import __appname__
@ -264,11 +264,11 @@ class ToolBar(QToolBar): # {{{
def apply_settings(self):
sz = gprefs['toolbar_icon_size']
sz = {'small':24, 'medium':48, 'large':64}[sz]
sz = {'off':0, 'small':24, 'medium':48, 'large':64}[sz]
self.setIconSize(QSize(sz, sz))
self.child_bar.setIconSize(QSize(sz, sz))
style = Qt.ToolButtonTextUnderIcon
if gprefs['toolbar_text'] == 'never':
if sz > 0 and gprefs['toolbar_text'] == 'never':
style = Qt.ToolButtonIconOnly
self.setToolButtonStyle(style)
self.child_bar.setToolButtonStyle(style)

4
src/calibre/gui2/library/delegates.py
View File

@ -353,7 +353,7 @@ class CcBoolDelegate(QStyledItemDelegate): # {{{
editor = DelegateCB(parent)
items = [_('Y'), _('N'), ' ']
icons = [I('ok.png'), I('list_remove.png'), I('blank.png')]
if tweaks['bool_custom_columns_are_tristate'] == 'no':
if not index.model().db.prefs.get('bools_are_tristate'):
items = items[:-1]
icons = icons[:-1]
for icon, text in zip(icons, items):
@ -367,7 +367,7 @@ class CcBoolDelegate(QStyledItemDelegate): # {{{
def setEditorData(self, editor, index):
m = index.model()
val = m.db.data[index.row()][m.custom_columns[m.column_map[index.column()]]['rec_index']]
if tweaks['bool_custom_columns_are_tristate'] == 'no':
if not m.db.prefs.get('bools_are_tristate'):
val = 1 if not val else 0
else:
val = 2 if val is None else 1 if not val else 0

4
src/calibre/gui2/library/models.py
View File

@ -700,7 +700,7 @@ class BooksModel(QAbstractTableModel): # {{{
self.dc_decorator[col] = functools.partial(
bool_type_decorator, idx=idx,
bool_cols_are_tristate=
tweaks['bool_custom_columns_are_tristate'] != 'no')
self.db.prefs.get('bools_are_tristate'))
elif datatype in ('int', 'float'):
self.dc[col] = functools.partial(number_type, idx=idx)
elif datatype == 'datetime':
@ -710,7 +710,7 @@ class BooksModel(QAbstractTableModel): # {{{
self.dc_decorator[col] = functools.partial(
bool_type_decorator, idx=idx,
bool_cols_are_tristate=
tweaks['bool_custom_columns_are_tristate'] != 'no')
self.db.prefs.get('bools_are_tristate'))
elif datatype == 'rating':
self.dc[col] = functools.partial(rating_type, idx=idx)
elif datatype == 'series':

72
src/calibre/gui2/metadata/basic_widgets.py
View File

@ -1,5 +1,7 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
@ -7,10 +9,10 @@ __docformat__ = 'restructuredtext en'
import textwrap, re, os
from PyQt4.Qt import Qt, QDateEdit, QDate, \
QIcon, QToolButton, QWidget, QLabel, QGridLayout, \
QDoubleSpinBox, QListWidgetItem, QSize, QPixmap, \
QPushButton, QSpinBox, QLineEdit
from PyQt4.Qt import (Qt, QDateEdit, QDate,
QIcon, QToolButton, QWidget, QLabel, QGridLayout,
QDoubleSpinBox, QListWidgetItem, QSize, QPixmap,
QPushButton, QSpinBox, QLineEdit, QSizePolicy)
from calibre.gui2.widgets import EnLineEdit, FormatList, ImageView
from calibre.gui2.complete import MultiCompleteLineEdit, MultiCompleteComboBox
@ -22,7 +24,7 @@ from calibre.ebooks.metadata.meta import get_metadata
from calibre.gui2 import file_icon_provider, UNDEFINED_QDATE, UNDEFINED_DATE, \
choose_files, error_dialog, choose_images, question_dialog
from calibre.utils.date import local_tz, qt_to_dt
from calibre import strftime
from calibre import strftime, fit_image
from calibre.ebooks import BOOK_EXTENSIONS
from calibre.customize.ui import run_plugins_on_import
from calibre.utils.date import utcfromtimestamp
@ -426,7 +428,7 @@ class Format(QListWidgetItem): # {{{
if timestamp is not None:
ts = timestamp.astimezone(local_tz)
t = strftime('%a, %d %b %Y [%H:%M:%S]', ts.timetuple())
text = _('Last modified: %s')%t
text = _('Last modified: %s\n\nDouble click to view')%t
self.setToolTip(text)
self.setStatusTip(text)
@ -480,6 +482,7 @@ class FormatsManager(QWidget): # {{{
def initialize(self, db, id_):
self.changed = False
self.formats.clear()
exts = db.formats(id_, index_is_id=True)
self.original_val = set([])
if exts:
@ -574,8 +577,7 @@ class FormatsManager(QWidget): # {{{
self.changed = True
def show_format(self, item, *args):
fmt = item.ext
self.dialog.view_format.emit(fmt)
self.dialog.do_view_format(item.path, item.ext)
def get_selected_format_metadata(self, db, id_):
old = prefs['read_file_metadata']
@ -638,6 +640,23 @@ class Cover(ImageView): # {{{
self.trim_cover_button, self.download_cover_button,
self.generate_cover_button]
self.frame_size = (300, 400)
self.setSizePolicy(QSizePolicy(QSizePolicy.Preferred,
QSizePolicy.Preferred))
def frame_resized(self, ev):
sz = ev.size()
self.frame_size = (sz.width()//3, sz.height())
def sizeHint(self):
sz = ImageView.sizeHint(self)
w, h = sz.width(), sz.height()
resized, nw, nh = fit_image(w, h, self.frame_size[0],
self.frame_size[1])
if resized:
sz = QSize(nw, nh)
return sz
def select_cover(self, *args):
files = choose_images(self, 'change cover dialog',
_('Choose cover for ') +
@ -882,8 +901,11 @@ class TagsEdit(MultiCompleteLineEdit): # {{{
# }}}
class ISBNEdit(QLineEdit): # {{{
LABEL = _('IS&BN:')
class IdentifiersEdit(QLineEdit): # {{{
LABEL = _('I&ds:')
BASE_TT = _('Edit the identifiers for this book. '
'For example: \n\n%s')%(
'isbn:1565927249, doi:10.1000/182, amazon:1565927249')
def __init__(self, parent):
QLineEdit.__init__(self, parent)
@ -893,32 +915,44 @@ class ISBNEdit(QLineEdit): # {{{
@dynamic_property
def current_val(self):
def fget(self):
return self.pat.sub('', unicode(self.text()).strip())
raw = unicode(self.text()).strip()
parts = [x.strip() for x in raw.split(',')]
ans = {}
for x in parts:
c = x.split(':')
if len(c) == 2:
ans[c[0]] = c[1]
return ans
def fset(self, val):
if not val:
val = ''
self.setText(val.strip())
val = {}
txt = ', '.join(['%s:%s'%(k, v) for k, v in val.iteritems()])
self.setText(txt.strip())
return property(fget=fget, fset=fset)
def initialize(self, db, id_):
self.current_val = db.isbn(id_, index_is_id=True)
self.current_val = db.get_identifiers(id_, index_is_id=True)
self.original_val = self.current_val
def commit(self, db, id_):
db.set_isbn(id_, self.current_val, notify=False, commit=False)
if self.original_val != self.current_val:
db.set_identifiers(id_, self.current_val, notify=False, commit=False)
return True
def validate(self, *args):
isbn = self.current_val
tt = _('This ISBN number is valid')
identifiers = self.current_val
isbn = identifiers.get('isbn', '')
tt = self.BASE_TT
extra = ''
if not isbn:
col = 'rgba(0,255,0,0%)'
elif check_isbn(isbn) is not None:
col = 'rgba(0,255,0,20%)'
extra = '\n\n'+_('This ISBN number is valid')
else:
col = 'rgba(255,0,0,20%)'
tt = _('This ISBN number is invalid')
self.setToolTip(tt)
extra = '\n\n' + _('This ISBN number is invalid')
self.setToolTip(tt+extra)
self.setStyleSheet('QLineEdit { background-color: %s }'%col)
# }}}

85
src/calibre/gui2/metadata/single.py
View File

@ -1,5 +1,7 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
@ -8,31 +10,31 @@ __docformat__ = 'restructuredtext en'
import os
from functools import partial
from PyQt4.Qt import Qt, QVBoxLayout, QHBoxLayout, QWidget, QPushButton, \
QGridLayout, pyqtSignal, QDialogButtonBox, QScrollArea, QFont, \
QTabWidget, QIcon, QToolButton, QSplitter, QGroupBox, QSpacerItem, \
QSizePolicy, QPalette, QFrame, QSize, QKeySequence
from PyQt4.Qt import (Qt, QVBoxLayout, QHBoxLayout, QWidget, QPushButton,
QGridLayout, pyqtSignal, QDialogButtonBox, QScrollArea, QFont,
QTabWidget, QIcon, QToolButton, QSplitter, QGroupBox, QSpacerItem,
QSizePolicy, QPalette, QFrame, QSize, QKeySequence)
from calibre.ebooks.metadata import authors_to_string, string_to_authors
from calibre.gui2 import ResizableDialog, error_dialog, gprefs
from calibre.gui2.metadata.basic_widgets import TitleEdit, AuthorsEdit, \
AuthorSortEdit, TitleSortEdit, SeriesEdit, SeriesIndexEdit, ISBNEdit, \
RatingEdit, PublisherEdit, TagsEdit, FormatsManager, Cover, CommentsEdit, \
BuddyLabel, DateEdit, PubdateEdit
from calibre.gui2.metadata.basic_widgets import (TitleEdit, AuthorsEdit,
AuthorSortEdit, TitleSortEdit, SeriesEdit, SeriesIndexEdit, IdentifiersEdit,
RatingEdit, PublisherEdit, TagsEdit, FormatsManager, Cover, CommentsEdit,
BuddyLabel, DateEdit, PubdateEdit)
from calibre.gui2.custom_column_widgets import populate_metadata_page
from calibre.utils.config import tweaks
class MetadataSingleDialogBase(ResizableDialog):
view_format = pyqtSignal(object)
view_format = pyqtSignal(object, object)
cc_two_column = tweaks['metadata_single_use_2_cols_for_custom_fields']
one_line_comments_toolbar = False
def __init__(self, db, parent=None):
self.db = db
self.changed = set([])
self.books_to_refresh = set([])
self.rows_to_refresh = set([])
self.changed = set()
self.books_to_refresh = set()
self.rows_to_refresh = set()
ResizableDialog.__init__(self, parent)
def setupUi(self, *args): # {{{
@ -145,8 +147,8 @@ class MetadataSingleDialogBase(ResizableDialog):
self.tags_editor_button.clicked.connect(self.tags_editor)
self.basic_metadata_widgets.append(self.tags)
self.isbn = ISBNEdit(self)
self.basic_metadata_widgets.append(self.isbn)
self.identifiers = IdentifiersEdit(self)
self.basic_metadata_widgets.append(self.identifiers)
self.publisher = PublisherEdit(self)
self.basic_metadata_widgets.append(self.publisher)
@ -192,6 +194,13 @@ class MetadataSingleDialogBase(ResizableDialog):
pass # Do something
# }}}
def do_view_format(self, path, fmt):
if path:
self.view_format.emit(None, path)
else:
self.view_format.emit(self.book_id, fmt)
def do_layout(self):
raise NotImplementedError()
@ -202,6 +211,8 @@ class MetadataSingleDialogBase(ResizableDialog):
widget.initialize(self.db, id_)
for widget in getattr(self, 'custom_metadata_widgets', []):
widget.initialize(id_)
if callable(self.set_current_callback):
self.set_current_callback(id_)
# Commented out as it doesn't play nice with Next, Prev buttons
#self.fetch_metadata_button.setFocus(Qt.OtherFocusReason)
@ -280,8 +291,8 @@ class MetadataSingleDialogBase(ResizableDialog):
self.publisher.current_val = mi.publisher
if not mi.is_null('tags'):
self.tags.current_val = mi.tags
if not mi.is_null('isbn'):
self.isbn.current_val = mi.isbn
if not mi.is_null('identifiers'):
self.identifiers.current_val = mi.identifiers
if not mi.is_null('pubdate'):
self.pubdate.current_val = mi.pubdate
if not mi.is_null('series') and mi.series.strip():
@ -337,11 +348,13 @@ class MetadataSingleDialogBase(ResizableDialog):
gprefs['metasingle_window_geometry3'] = bytearray(self.saveGeometry())
# Dialog use methods {{{
def start(self, row_list, current_row, view_slot=None):
def start(self, row_list, current_row, view_slot=None,
set_current_callback=None):
self.row_list = row_list
self.current_row = current_row
if view_slot is not None:
self.view_format.connect(view_slot)
self.set_current_callback = set_current_callback
self.do_one(apply_changes=False)
ret = self.exec_()
self.break_cycles()
@ -373,6 +386,7 @@ class MetadataSingleDialogBase(ResizableDialog):
def break_cycles(self):
# Break any reference cycles that could prevent python
# from garbage collecting this dialog
self.set_current_callback = self.db = None
def disconnect(signal):
try:
signal.disconnect()
@ -385,6 +399,14 @@ class MetadataSingleDialogBase(ResizableDialog):
disconnect(x.clicked)
# }}}
class Splitter(QSplitter):
frame_resized = pyqtSignal(object)
def resizeEvent(self, ev):
self.frame_resized.emit(ev)
return QSplitter.resizeEvent(self, ev)
class MetadataSingleDialog(MetadataSingleDialogBase): # {{{
def do_layout(self):
@ -437,8 +459,9 @@ class MetadataSingleDialog(MetadataSingleDialogBase): # {{{
tl.addWidget(self.formats_manager, 0, 6, 3, 1)
self.splitter = QSplitter(Qt.Horizontal, self)
self.splitter = Splitter(Qt.Horizontal, self)
self.splitter.addWidget(self.cover)
self.splitter.frame_resized.connect(self.cover.frame_resized)
l.addWidget(self.splitter)
self.tabs[0].gb = gb = QGroupBox(_('Change cover'), self)
gb.l = l = QGridLayout()
@ -475,9 +498,9 @@ class MetadataSingleDialog(MetadataSingleDialogBase): # {{{
create_row2(1, self.rating)
sto(self.rating, self.tags)
create_row2(2, self.tags, self.tags_editor_button)
sto(self.tags_editor_button, self.isbn)
create_row2(3, self.isbn)
sto(self.isbn, self.timestamp)
sto(self.tags_editor_button, self.identifiers)
create_row2(3, self.identifiers)
sto(self.identifiers, self.timestamp)
create_row2(4, self.timestamp, self.timestamp.clear_button)
sto(self.timestamp.clear_button, self.pubdate)
create_row2(5, self.pubdate, self.pubdate.clear_button)
@ -498,7 +521,7 @@ class MetadataSingleDialog(MetadataSingleDialogBase): # {{{
# }}}
class MetadataSingleDialogAlt(MetadataSingleDialogBase): # {{{
class MetadataSingleDialogAlt1(MetadataSingleDialogBase): # {{{
cc_two_column = False
one_line_comments_toolbar = True
@ -562,9 +585,9 @@ class MetadataSingleDialogAlt(MetadataSingleDialogBase): # {{{
create_row(8, self.pubdate, self.publisher,
button=self.pubdate.clear_button, icon='trash.png')
create_row(9, self.publisher, self.timestamp)
create_row(10, self.timestamp, self.isbn,
create_row(10, self.timestamp, self.identifiers,
button=self.timestamp.clear_button, icon='trash.png')
create_row(11, self.isbn, self.comments)
create_row(11, self.identifiers, self.comments)
tl.addItem(QSpacerItem(1, 1, QSizePolicy.Fixed, QSizePolicy.Expanding),
12, 1, 1 ,1)
@ -580,7 +603,7 @@ class MetadataSingleDialogAlt(MetadataSingleDialogBase): # {{{
sr.setWidget(w)
gbl.addWidget(sr)
self.tabs[0].l.addWidget(gb, 0, 1, 1, 1)
sto(self.isbn, gb)
sto(self.identifiers, gb)
w = QGroupBox(_('&Comments'), tab0)
sp = QSizePolicy()
@ -631,10 +654,16 @@ class MetadataSingleDialogAlt(MetadataSingleDialogBase): # {{{
# }}}
editors = {'default': MetadataSingleDialog, 'alt1': MetadataSingleDialogAlt1}
def edit_metadata(db, row_list, current_row, parent=None, view_slot=None):
d = MetadataSingleDialog(db, parent)
d.start(row_list, current_row, view_slot=view_slot)
def edit_metadata(db, row_list, current_row, parent=None, view_slot=None,
set_current_callback=None):
cls = db.prefs.get('edit_metadata_single_layout', '')
if cls not in editors:
cls = 'default'
d = editors[cls](db, parent)
d.start(row_list, current_row, view_slot=view_slot,
set_current_callback=set_current_callback)
return d.changed, d.rows_to_refresh
if __name__ == '__main__':

465
src/calibre/gui2/metadata/single_download.py Normal file
View File

@ -0,0 +1,465 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
from threading import Thread, Event
from operator import attrgetter
from PyQt4.Qt import (QStyledItemDelegate, QTextDocument, QRectF, QIcon, Qt,
QStyle, QApplication, QDialog, QVBoxLayout, QLabel, QDialogButtonBox,
QStackedWidget, QWidget, QTableView, QGridLayout, QFontInfo, QPalette,
QTimer, pyqtSignal, QAbstractTableModel, QVariant, QSize)
from PyQt4.QtWebKit import QWebView
from calibre.customize.ui import metadata_plugins
from calibre.ebooks.metadata import authors_to_string
from calibre.utils.logging import GUILog as Log
from calibre.ebooks.metadata.sources.identify import identify
from calibre.ebooks.metadata.book.base import Metadata
from calibre.gui2 import error_dialog, NONE
from calibre.utils.date import utcnow, fromordinal, format_date
from calibre.library.comments import comments_to_html
class RichTextDelegate(QStyledItemDelegate): # {{{
def __init__(self, parent=None):
QStyledItemDelegate.__init__(self, parent)
def to_doc(self, index):
doc = QTextDocument()
doc.setHtml(index.data().toString())
return doc
def sizeHint(self, option, index):
ans = self.to_doc(index).size().toSize()
ans.setHeight(ans.height()+10)
return ans
def paint(self, painter, option, index):
painter.save()
painter.setClipRect(QRectF(option.rect))
if hasattr(QStyle, 'CE_ItemViewItem'):
QApplication.style().drawControl(QStyle.CE_ItemViewItem, option, painter)
elif option.state & QStyle.State_Selected:
painter.fillRect(option.rect, option.palette.highlight())
painter.translate(option.rect.topLeft())
self.to_doc(index).drawContents(painter)
painter.restore()
# }}}
class ResultsModel(QAbstractTableModel): # {{{
COLUMNS = (
'#', _('Title'), _('Published'), _('Has cover'), _('Has summary')
)
HTML_COLS = (1, 2)
ICON_COLS = (3, 4)
def __init__(self, results, parent=None):
QAbstractTableModel.__init__(self, parent)
self.results = results
self.yes_icon = QVariant(QIcon(I('ok.png')))
def rowCount(self, parent=None):
return len(self.results)
def columnCount(self, parent=None):
return len(self.COLUMNS)
def headerData(self, section, orientation, role):
if orientation == Qt.Horizontal and role == Qt.DisplayRole:
try:
return QVariant(self.COLUMNS[section])
except:
return NONE
return NONE
def data_as_text(self, book, col):
if col == 0:
return unicode(book.gui_rank+1)
if col == 1:
t = book.title if book.title else _('Unknown')
a = authors_to_string(book.authors) if book.authors else ''
return '<b>%s</b><br><i>%s</i>' % (t, a)
if col == 2:
d = format_date(book.pubdate, 'yyyy') if book.pubdate else _('Unknown')
p = book.publisher if book.publisher else ''
return '<b>%s</b><br><i>%s</i>' % (d, p)
def data(self, index, role):
row, col = index.row(), index.column()
try:
book = self.results[row]
except:
return NONE
if role == Qt.DisplayRole and col not in self.ICON_COLS:
res = self.data_as_text(book, col)
if res:
return QVariant(res)
return NONE
elif role == Qt.DecorationRole and col in self.ICON_COLS:
if col == 3 and getattr(book, 'has_cached_cover_url', False):
return self.yes_icon
if col == 4 and book.comments:
return self.yes_icon
elif role == Qt.UserRole:
return book
return NONE
def sort(self, col, order=Qt.AscendingOrder):
key = lambda x: x
if col == 0:
key = attrgetter('gui_rank')
elif col == 1:
key = attrgetter('title')
elif col == 2:
key = attrgetter('authors')
elif col == 3:
key = attrgetter('has_cached_cover_url')
elif key == 4:
key = lambda x: bool(x.comments)
self.results.sort(key=key, reverse=order==Qt.AscendingOrder)
self.reset()
# }}}
class ResultsView(QTableView): # {{{
show_details_signal = pyqtSignal(object)
book_selected = pyqtSignal(object)
def __init__(self, parent=None):
QTableView.__init__(self, parent)
self.rt_delegate = RichTextDelegate(self)
self.setSelectionMode(self.SingleSelection)
self.setAlternatingRowColors(True)
self.setSelectionBehavior(self.SelectRows)
self.setIconSize(QSize(24, 24))
self.clicked.connect(self.show_details)
self.doubleClicked.connect(self.select_index)
self.setSortingEnabled(True)
def show_results(self, results):
self._model = ResultsModel(results, self)
self.setModel(self._model)
for i in self._model.HTML_COLS:
self.setItemDelegateForColumn(i, self.rt_delegate)
self.resizeRowsToContents()
self.resizeColumnsToContents()
self.setFocus(Qt.OtherFocusReason)
def currentChanged(self, current, previous):
ret = QTableView.currentChanged(self, current, previous)
self.show_details(current)
return ret
def show_details(self, index):
book = self.model().data(index, Qt.UserRole)
parts = [
'<center>',
'<h2>%s</h2>'%book.title,
'<div><i>%s</i></div>'%authors_to_string(book.authors),
]
if not book.is_null('rating'):
parts.append('<div>%s</div>'%('\u2605'*int(book.rating)))
parts.append('</center>')
if book.tags:
parts.append('<div>%s</div><div>\u00a0</div>'%', '.join(book.tags))
if book.comments:
parts.append(comments_to_html(book.comments))
self.show_details_signal.emit(''.join(parts))
def select_index(self, index):
if not index.isValid():
index = self.model().index(0, 0)
book = self.model().data(index, Qt.UserRole)
self.book_selected.emit(book)
def get_result(self):
self.select_index(self.currentIndex())
# }}}
class Comments(QWebView): # {{{
def __init__(self, parent=None):
QWebView.__init__(self, parent)
self.setAcceptDrops(False)
self.setMaximumWidth(300)
self.setMinimumWidth(300)
palette = self.palette()
palette.setBrush(QPalette.Base, Qt.transparent)
self.page().setPalette(palette)
self.setAttribute(Qt.WA_OpaquePaintEvent, False)
def turnoff_scrollbar(self, *args):
self.page().mainFrame().setScrollBarPolicy(Qt.Horizontal, Qt.ScrollBarAlwaysOff)
def show_data(self, html):
def color_to_string(col):
ans = '#000000'
if col.isValid():
col = col.toRgb()
if col.isValid():
ans = unicode(col.name())
return ans
f = QFontInfo(QApplication.font(self.parent())).pixelSize()
c = color_to_string(QApplication.palette().color(QPalette.Normal,
QPalette.WindowText))
templ = '''\
<html>
<head>
<style type="text/css">
body, td {background-color: transparent; font-size: %dpx; color: %s }
a { text-decoration: none; color: blue }
div.description { margin-top: 0; padding-top: 0; text-indent: 0 }
table { margin-bottom: 0; padding-bottom: 0; }
</style>
</head>
<body>
<div class="description">
%%s
</div>
</body>
<html>
'''%(f, c)
self.setHtml(templ%html)
# }}}
class IdentifyWorker(Thread): # {{{
def __init__(self, log, abort, title, authors, identifiers):
Thread.__init__(self)
self.daemon = True
self.log, self.abort = log, abort
self.title, self.authors, self.identifiers = (title, authors,
identifiers)
self.results = []
self.error = None
def sample_results(self):
m1 = Metadata('The Great Gatsby', ['Francis Scott Fitzgerald'])
m2 = Metadata('The Great Gatsby', ['F. Scott Fitzgerald'])
m1.has_cached_cover_url = True
m2.has_cached_cover_url = False
m1.comments = 'Some comments '*10
m1.tags = ['tag%d'%i for i in range(20)]
m1.rating = 4.4
m1.language = 'en'
m2.language = 'fr'
m1.pubdate = utcnow()
m2.pubdate = fromordinal(1000000)
m1.publisher = 'Publisher 1'
m2.publisher = 'Publisher 2'
return [m1, m2]
def run(self):
try:
if True:
self.results = self.sample_results()
else:
self.results = identify(self.log, self.abort, title=self.title,
authors=self.authors, identifiers=self.identifiers)
for i, result in enumerate(self.results):
result.gui_rank = i
except:
import traceback
self.error = traceback.format_exc()
# }}}
class IdentifyWidget(QWidget): # {{{
rejected = pyqtSignal()
results_found = pyqtSignal()
book_selected = pyqtSignal(object)
def __init__(self, log, parent=None):
QWidget.__init__(self, parent)
self.log = log
self.abort = Event()
self.l = l = QGridLayout()
self.setLayout(l)
names = ['<b>'+p.name+'</b>' for p in metadata_plugins(['identify']) if
p.is_configured()]
self.top = QLabel('<p>'+_('calibre is downloading metadata from: ') +
', '.join(names))
self.top.setWordWrap(True)
l.addWidget(self.top, 0, 0)
self.results_view = ResultsView(self)
self.results_view.book_selected.connect(self.book_selected.emit)
self.get_result = self.results_view.get_result
l.addWidget(self.results_view, 1, 0)
self.comments_view = Comments(self)
l.addWidget(self.comments_view, 1, 1)
self.results_view.show_details_signal.connect(self.comments_view.show_data)
self.query = QLabel('download starting...')
f = self.query.font()
f.setPointSize(f.pointSize()-2)
self.query.setFont(f)
self.query.setWordWrap(True)
l.addWidget(self.query, 2, 0, 1, 2)
self.comments_view.show_data('<h2>'+_('Downloading')+
'<br><span id="dots">.</span></h2>'+
'''
<script type="text/javascript">
window.onload=function(){
var dotspan = document.getElementById('dots');
window.setInterval(function(){
if(dotspan.textContent == '............'){
dotspan.textContent = '.';
}
else{
dotspan.textContent += '.';
}
}, 400);
}
</script>
''')
def start(self, title=None, authors=None, identifiers={}):
self.log.clear()
self.log('Starting download')
parts = []
if title:
parts.append('title:'+title)
if authors:
parts.append('authors:'+authors_to_string(authors))
if identifiers:
x = ', '.join('%s:%s'%(k, v) for k, v in identifiers)
parts.append(x)
self.query.setText(_('Query: ')+'; '.join(parts))
self.log(unicode(self.query.text()))
self.worker = IdentifyWorker(self.log, self.abort, title,
authors, identifiers)
self.worker.start()
QTimer.singleShot(50, self.update)
def update(self):
if self.worker.is_alive():
QTimer.singleShot(50, self.update)
else:
self.process_results()
def process_results(self):
if self.worker.error is not None:
error_dialog(self, _('Download failed'),
_('Failed to download metadata. Click '
'Show Details to see details'),
show=True, det_msg=self.worker.error)
self.rejected.emit()
return
if not self.worker.results:
log = ''.join(self.log.plain_text)
error_dialog(self, _('No matches found'), '<p>' +
_('Failed to find any books that '
'match your search. Try making the search <b>less '
'specific</b>. For example, use only the author\'s '
'last name and a single distinctive word from '
'the title.<p>To see the full log, click Show Details.'),
show=True, det_msg=log)
self.rejected.emit()
return
self.results_view.show_results(self.worker.results)
self.comments_view.show_data('''
<div style="margin-bottom:2ex">Found <b>%d</b> results</div>
<div>To see <b>details</b>, click on any result</div>''' %
len(self.worker.results))
self.results_found.emit()
def cancel(self):
self.abort.set()
# }}}
class FullFetch(QDialog): # {{{
def __init__(self, log, parent=None):
QDialog.__init__(self, parent)
self.log = log
self.setWindowTitle(_('Downloading metadata...'))
self.setWindowIcon(QIcon(I('metadata.png')))
self.stack = QStackedWidget()
self.l = l = QVBoxLayout()
self.setLayout(l)
l.addWidget(self.stack)
self.bb = QDialogButtonBox(QDialogButtonBox.Cancel|QDialogButtonBox.Ok)
l.addWidget(self.bb)
self.bb.rejected.connect(self.reject)
self.next_button = self.bb.addButton(_('Next'), self.bb.AcceptRole)
self.next_button.setDefault(True)
self.next_button.setEnabled(False)
self.next_button.clicked.connect(self.next_clicked)
self.ok_button = self.bb.button(self.bb.Ok)
self.ok_button.setVisible(False)
self.ok_button.clicked.connect(self.ok_clicked)
self.identify_widget = IdentifyWidget(log, self)
self.identify_widget.rejected.connect(self.reject)
self.identify_widget.results_found.connect(self.identify_results_found)
self.identify_widget.book_selected.connect(self.book_selected)
self.stack.addWidget(self.identify_widget)
self.resize(850, 500)
def book_selected(self, book):
print (book)
self.next_button.setVisible(False)
self.ok_button.setVisible(True)
def accept(self):
# Prevent the usual dialog accept mechanisms from working
pass
def reject(self):
self.identify_widget.cancel()
return QDialog.reject(self)
def identify_results_found(self):
self.next_button.setEnabled(True)
def next_clicked(self, *args):
self.identify_widget.get_result()
def ok_clicked(self, *args):
pass
def start(self, title=None, authors=None, identifiers={}):
self.identify_widget.start(title=title, authors=authors,
identifiers=identifiers)
self.exec_()
# }}}
if __name__ == '__main__':
app = QApplication([])
d = FullFetch(Log())
d.start(title='great gatsby', authors=['Fitzgerald'])

13
src/calibre/gui2/preferences/columns.py
View File

@ -13,6 +13,7 @@ from calibre.gui2.preferences import ConfigWidgetBase, test_widget
from calibre.gui2.preferences.columns_ui import Ui_Form
from calibre.gui2.preferences.create_custom_column import CreateCustomColumn
from calibre.gui2 import error_dialog, question_dialog, ALL_COLUMNS
from calibre.utils.config import test_eight_code
class ConfigWidget(ConfigWidgetBase, Ui_Form):
@ -33,6 +34,14 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
signal = getattr(self.opt_columns, 'item'+signal)
signal.connect(self.columns_changed)
if test_eight_code:
r = self.register
choices = [(_('Default'), 'default'), (_('Compact Metadata'), 'alt1')]
r('edit_metadata_single_layout', db.prefs, choices=choices)
r('bools_are_tristate', db.prefs, restart_required=True)
else:
self.items_in_v_eight.setVisible(False)
def initialize(self):
ConfigWidgetBase.initialize(self)
self.init_columns()
@ -169,6 +178,10 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
must_restart = True
return must_restart
def refresh_gui(self, gui):
gui.library_view.reset()
if __name__ == '__main__':
from PyQt4.Qt import QApplication

61
src/calibre/gui2/preferences/columns.ui
View File

@ -197,6 +197,67 @@
</property>
</widget>
</item>
<item row="1" column="2">
<layout class="QVBoxLayout">
<item>
<widget class="QGroupBox" name="items_in_v_eight">
<property name="title">
<string>Related Options</string>
</property>
<layout class="QGridLayout" name="gridLayout">
<item row="0" column="0">
<widget class="QLabel">
<property name="text">
<string>Edit metadata layout:</string>
</property>
<property name="buddy">
<cstring>opt_edit_metadata_single_layout</cstring>
</property>
</widget>
</item>
<item row="0" column="1">
<widget class="QComboBox" name="opt_edit_metadata_single_layout">
<property name="toolTip">
<string>Choose a different layout for the Edit Metadata dialog. Alternate layouts make it easier to edit custom columns.</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLabel">
<property name="text">
<string>Boolean columns are tristate:</string>
</property>
<property name="buddy">
<cstring>opt_bools_are_tristate</cstring>
</property>
</widget>
</item>
<item row="1" column="1">
<widget class="QCheckBox" name="opt_bools_are_tristate">
<property name="toolTip">
<string>If checked, boolean columns values can be Yes, No, and Unknown.
If not checked, the values can be Yes and No.</string>
</property>
</widget>
</item>
</layout>
</widget>
</item>
<item>
<spacer name="verticalSpacer_5">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>40</height>
</size>
</property>
</spacer>
</item>
</layout>
</item>
</layout>
</widget>
<resources>

4
src/calibre/gui2/preferences/look_feel.py
View File

@ -49,8 +49,8 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
r('use_roman_numerals_for_series_number', config)
r('separate_cover_flow', config, restart_required=True)
choices = [(_('Small'), 'small'), (_('Medium'), 'medium'),
(_('Large'), 'large')]
choices = [(_('Off'), 'off'), (_('Small'), 'small'),
(_('Medium'), 'medium'), (_('Large'), 'large')]
r('toolbar_icon_size', gprefs, choices=choices)
choices = [(_('Automatic'), 'auto'), (_('Always'), 'always'),

19
src/calibre/gui2/preferences/server.py
View File

@ -18,6 +18,7 @@ from calibre.utils.config import ConfigProxy
from calibre.gui2 import error_dialog, config, open_url, warning_dialog, \
Dispatcher, info_dialog
from calibre import as_unicode
from calibre.utils.icu import sort_key
class ConfigWidget(ConfigWidgetBase, Ui_Form):
@ -42,8 +43,11 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
else self.opt_password.Password))
self.opt_password.setEchoMode(self.opt_password.Password)
restrictions = sorted(saved_searches().names(),
cmp=lambda x,y: cmp(x.lower(), y.lower()))
restrictions = sorted(saved_searches().names(), key=sort_key)
# verify that the current restriction still exists. If not, clear it.
csr = db.prefs.get('cs_restriction', None)
if csr and csr not in restrictions:
db.prefs.set('cs_restriction', '')
choices = [('', '')] + [(x, x) for x in restrictions]
r('cs_restriction', db.prefs, choices=choices)
@ -57,17 +61,8 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
r('autolaunch_server', config)
def set_server_options(self):
c = self.proxy
c.set('port', self.opt_port.value())
c.set('username', unicode(self.opt_username.text()).strip())
p = unicode(self.opt_password.text()).strip()
if not p:
p = None
c.set('password', p)
def start_server(self):
self.set_server_options()
ConfigWidgetBase.commit(self)
self.gui.start_content_server(check_started=False)
while not self.gui.content_server.is_running and self.gui.content_server.exception is None:
time.sleep(1)

1
src/calibre/gui2/tag_view.py
View File

@ -985,6 +985,7 @@ class TagsModel(QAbstractItemModel): # {{{
def do_drop_from_library(self, md, action, row, column, parent):
idx = parent
if idx.isValid():
self.tags_view.setCurrentIndex(idx)
node = self.data(idx, Qt.UserRole)
if node.type == TagTreeItem.TAG:
fm = self.db.metadata_for_field(node.tag.category)

1
src/calibre/gui2/widgets.py
View File

@ -312,6 +312,7 @@ class ImageView(QWidget, ImageDropMixin):
p.setPen(pen)
if self.draw_border:
p.drawRect(target)
#p.drawRect(self.rect())
p.end()
class CoverView(QGraphicsView, ImageDropMixin):

9
src/calibre/library/caches.py
View File

@ -547,7 +547,7 @@ class ResultCache(SearchQueryParser): # {{{
return matchkind, query
def get_bool_matches(self, location, query, candidates):
bools_are_tristate = tweaks['bool_custom_columns_are_tristate'] != 'no'
bools_are_tristate = not self.db_prefs.get('bools_are_tristate')
loc = self.field_metadata[location]['rec_index']
matches = set()
query = icu_lower(query)
@ -947,7 +947,7 @@ class ResultCache(SearchQueryParser): # {{{
if not fields:
fields = [('timestamp', False)]
keyg = SortKeyGenerator(fields, self.field_metadata, self._data)
keyg = SortKeyGenerator(fields, self.field_metadata, self._data, self.db_prefs)
self._map.sort(key=keyg)
tmap = list(itertools.repeat(False, len(self._data)))
@ -970,9 +970,10 @@ class SortKey(object):
class SortKeyGenerator(object):
def __init__(self, fields, field_metadata, data):
def __init__(self, fields, field_metadata, data, db_prefs):
from calibre.utils.icu import sort_key
self.field_metadata = field_metadata
self.db_prefs = db_prefs
self.orders = [1 if x[1] else -1 for x in fields]
self.entries = [(x[0], field_metadata[x[0]]) for x in fields]
self.library_order = tweaks['title_series_sorting'] == 'library_order'
@ -1032,7 +1033,7 @@ class SortKeyGenerator(object):
val = self.string_sort_key(val)
elif dt == 'bool':
if tweaks['bool_custom_columns_are_tristate'] == 'no':
if not self.db_prefs.get('bools_are_tristate'):
val = {True: 1, False: 2, None: 2}.get(val, 2)
else:
val = {True: 1, False: 2, None: 3}.get(val, 3)

8
src/calibre/library/database2.py
View File

@ -40,7 +40,7 @@ from calibre.ebooks import BOOK_EXTENSIONS, check_ebook_format
from calibre.utils.magick.draw import save_cover_data_to
from calibre.utils.recycle_bin import delete_file, delete_tree
from calibre.utils.formatter_functions import load_user_template_functions
from calibre.utils.config import test_eight_code
copyfile = os.link if hasattr(os, 'link') else shutil.copyfile
@ -213,6 +213,12 @@ class LibraryDatabase2(LibraryDatabase, SchemaUpgrade, CustomColumns):
defs = self.prefs.defaults
defs['gui_restriction'] = defs['cs_restriction'] = ''
defs['categories_using_hierarchy'] = []
defs['edit_metadata_single_layout'] = 'default'
defs['bools_are_tristate'] = \
tweaks.get('bool_custom_columns_are_tristate', 'yes') == 'yes'
if self.prefs.get('bools_are_tristate') is None or not test_eight_code:
self.prefs.set('bools_are_tristate', defs['bools_are_tristate'])
# Migrate saved search and user categories to db preference scheme
def migrate_preference(key, default):

9
src/calibre/library/server/base.py
View File

@ -24,6 +24,8 @@ from calibre.library.server.xml import XMLServer
from calibre.library.server.opds import OPDSServer
from calibre.library.server.cache import Cache
from calibre.library.server.browse import BrowseServer
from calibre.utils.search_query_parser import saved_searches
from calibre import prints
class DispatchController(object): # {{{
@ -178,7 +180,12 @@ class LibraryServer(ContentServer, MobileServer, XMLServer, OPDSServer, Cache,
def set_search_restriction(self, restriction):
self.search_restriction_name = restriction
if restriction:
self.search_restriction = 'search:"%s"'%restriction
if restriction not in saved_searches().names():
prints('WARNING: Content server: search restriction ',
restriction, ' does not exist')
self.search_restriction = ''
else:
self.search_restriction = 'search:"%s"'%restriction
else:
self.search_restriction = ''
self.reset_caches()

7
src/calibre/library/server/content.py
View File

@ -17,8 +17,8 @@ from calibre.utils.magick.draw import save_cover_data_to, Image, \
class CSSortKeyGenerator(SortKeyGenerator):
def __init__(self, fields, fm):
SortKeyGenerator.__init__(self, fields, fm, None)
def __init__(self, fields, fm, db_prefs):
SortKeyGenerator.__init__(self, fields, fm, None, db_prefs)
def __call__(self, record):
return self.itervals(record).next()
@ -56,7 +56,8 @@ class ContentServer(object):
field = self.db.data.sanitize_sort_field_name(field)
if field not in self.db.field_metadata.sortable_field_keys():
raise cherrypy.HTTPError(400, '%s is not a valid sort field'%field)
keyg = CSSortKeyGenerator([(field, order)], self.db.field_metadata)
keyg = CSSortKeyGenerator([(field, order)], self.db.field_metadata,
self.db.prefs)
items.sort(key=keyg, reverse=not order)
# }}}

10
src/calibre/linux.py
View File

@ -30,7 +30,7 @@ entry_points = {
'calibre-customize = calibre.customize.ui:main',
'calibre-complete = calibre.utils.complete:main',
'pdfmanipulate = calibre.ebooks.pdf.manipulate.cli:main',
'fetch-ebook-metadata = calibre.ebooks.metadata.fetch:main',
'fetch-ebook-metadata = calibre.ebooks.metadata.sources.cli:main',
'epub-fix = calibre.ebooks.epub.fix.main:main',
'calibre-smtp = calibre.utils.smtp:main',
],
@ -136,17 +136,17 @@ class PostInstall:
self.icon_resources = []
self.menu_resources = []
self.mime_resources = []
if islinux:
if islinux or isfreebsd:
self.setup_completion()
self.install_man_pages()
if islinux:
if islinux or isfreebsd:
self.setup_desktop_integration()
self.create_uninstaller()
from calibre.utils.config import config_dir
if os.path.exists(config_dir):
os.chdir(config_dir)
if islinux:
if islinux or isfreebsd:
for f in os.listdir('.'):
if os.stat(f).st_uid == 0:
os.rmdir(f) if os.path.isdir(f) else os.unlink(f)
@ -183,7 +183,7 @@ class PostInstall:
from calibre.ebooks.lrf.lrfparser import option_parser as lrf2lrsop
from calibre.gui2.lrf_renderer.main import option_parser as lrfviewerop
from calibre.gui2.viewer.main import option_parser as viewer_op
from calibre.ebooks.metadata.fetch import option_parser as fem_op
from calibre.ebooks.metadata.sources.cli import option_parser as fem_op
from calibre.gui2.main import option_parser as guiop
from calibre.utils.smtp import option_parser as smtp_op
from calibre.library.server.main import option_parser as serv_op

2
src/calibre/manual/conf.py
View File

@ -126,7 +126,7 @@ html_use_modindex = False
html_use_index = False
# If true, the reST sources are included in the HTML build as _sources/<name>.
html_copy_source = False
html_copy_source = True
# Output file base name for HTML help builder.
htmlhelp_basename = 'calibredoc'

2
src/calibre/manual/creating_plugins.rst
View File

@ -197,7 +197,7 @@ Once you've located the zip file of your plugin you can then directly update it
zip -R /path/to/plugin/zip/file.zip *
This will automatically update all changed files. It relies on the freely available zip command line tool.
This will update all changed files. It relies on the freely available zip command line tool. Note that you should quit calibre before running this command.
More plugin examples
----------------------

3
src/calibre/manual/faq.rst
View File

@ -99,7 +99,8 @@ We just need some information from you:
device.
Once you send us the output for a particular operating system, support for the device in that operating system
will appear in the next release of |app|.
will appear in the next release of |app|. To send us the output, open a bug report and attach the output to it.
See `calibre bugs <http://calibre-ebook.com/bugs>`_.
My device is not being detected by |app|?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

3
src/calibre/manual/gui.rst
View File

@ -71,7 +71,7 @@ Edit metadata
|emii| The :guilabel:`Edit metadata` action has six variations, which can be accessed by clicking the down arrow on the right side of the button.
1. **Edit metadata individually**: This allows you to edit the metadata of books one-by-one, with the option of fetching metadata, including covers from the internet. It also allows you to add/remove particular ebook formats from a book. For more detail see :ref:`metadata`.
1. **Edit metadata individually**: This allows you to edit the metadata of books one-by-one, with the option of fetching metadata, including covers from the internet. It also allows you to add/remove particular ebook formats from a book.
2. **Edit metadata in bulk**: This allows you to edit common metadata fields for large numbers of books simulataneously. It operates on all the books you have selected in the :ref:`Library view <search_sort>`.
3. **Download metadata and covers**: Downloads metadata and covers (if available), for the books that are selected in the book list.
4. **Download only metadata**: Downloads only metadata (if available), for the books that are selected in the book list.
@ -79,6 +79,7 @@ Edit metadata
6. **Download only social metadata**: Downloads only social metadata such as tags and reviews (if available), for the books that are selected in the book list.
7. **Merge Book Records**: Gives you the capability of merging the metadata and formats of two or more book records together. You can choose to either delete or keep the records that were not clicked first.
For more details see :ref:`metadata`.
.. _convert_ebooks:

81
src/calibre/manual/index.rst
View File

@ -40,3 +40,84 @@ Sections
glossary
The main |app| user interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
gui
Adding your favorite news website to |app|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
news
The |app| e-book viewer
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
viewer
Customizing |app|'s e-book conversion
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
conversion
Editing e-book metadata
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
metadata
Frequently Asked Questions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
faq
Tutorials
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
tutorials
Customizing |app|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
customize
The Command Line Interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
cli/cli-index
Setting up a |app| development environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
:maxdepth: 2
develop

10
src/calibre/manual/news.rst
View File

@ -263,20 +263,18 @@ Tips for developing new recipes
The best way to develop new recipes is to use the command line interface. Create the recipe using your favorite python editor and save it to a file say :file:`myrecipe.recipe`. The `.recipe` extension is required. You can download content using this recipe with the command::
ebook-convert myrecipe.recipe output_dir --test -vv
ebook-convert myrecipe.recipe .epub --test -vv --debug-pipeline debug
The :command:`ebook-convert` will download all the webpages and save them to the directory :file:`output_dir`, creating it if necessary. The :option:`-vv` makes ebook-convert spit out a lot of information about what it is doing. The :option:`--test` makes it download only a couple of articles from at most two feeds.
The command :command:`ebook-convert` will download all the webpages and save them to the EPUB file :file:`myrecipe.epub`. The :option:`-vv` makes ebook-convert spit out a lot of information about what it is doing. The :option:`--test` makes it download only a couple of articles from at most two feeds. In addition, ebook-convert will put the downloaded HTML into the ``debug/input`` directory, where ``debug`` is the directory you specified in the :option:`--debug-pipeline` option.
Once the download is complete, you can look at the downloaded :term:`HTML` by opening the file :file:`index.html` in a browser. Once you're satisfied that the download and preprocessing is happening correctly, you can generate ebooks in different formats as shown below::
Once the download is complete, you can look at the downloaded :term:`HTML` by opening the file :file:`debug/input/index.html` in a browser. Once you're satisfied that the download and preprocessing is happening correctly, you can generate ebooks in different formats as shown below::
ebook-convert myrecipe.recipe myrecipe.epub
ebook-convert myrecipe.recipe myrecipe.mobi
...
If you're satisfied with your recipe, and you feel there is enough demand to justify its inclusion into the set of built-in recipes, add a comment to the ticket http://bugs.calibre-ebook.com/ticket/405
Alternatively, you could just post your recipe in the calibre forum at http://www.mobileread.com/forums/forumdisplay.php?f=166 to share it with other calibre users.
If you're satisfied with your recipe, and you feel there is enough demand to justify its inclusion into the set of built-in recipes, post your recipe in the `calibre recipes forum <http://www.mobileread.com/forums/forumdisplay.php?f=228>`_ to share it with other calibre users.
.. seealso::

4
src/calibre/manual/server.rst
View File

@ -16,7 +16,7 @@ Here, we will show you how to integrate the |app| content server into another se
Using a reverse proxy
-----------------------
This is the simplest approach as it allows you to use the binary calibre install with no external dependencies/system integration requirements.
A reverse proxy is when your normal server accepts incoming requests and passes them onto the calibre server. It then reads the response from the calibre server and forwards it to the client. This means that you can simply run the calibre server as normal without trying to integrate it closely with your main server, and you can take advantage of whatever authentication systems you main server has in place. This is the simplest approach as it allows you to use the binary calibre install with no external dependencies/system integration requirements. Below, is an example of how to achieve this with Apache as your main server, but it will work with any server that supports Reverse Proxies.
First start the |app| content server as shown below::
@ -33,7 +33,7 @@ The exact technique for enabling the proxy modules will vary depending on your A
RewriteRule ^/calibre/(.*) http://localhost:8080/calibre/$1 [proxy]
RewriteRule ^/calibre http://localhost:8080 [proxy]
That's all, you will now be able to access the |app| Content Server under the /calibre URL in your apache server.
That's all, you will now be able to access the |app| Content Server under the /calibre URL in your apache server. The above rules pass all requests under /calibre to the calibre server running on port 8080 and thanks to the --url-prefix option above, the calibre server handles them transparently.
.. note:: If you are willing to devote an entire VirtualHost to the content server, then there is no need to use --url-prefix and RewriteRule, instead just use the ProxyPass directive.

1083
src/calibre/translations/af.po
View File

File diff suppressed because it is too large Load Diff

1078
src/calibre/translations/ar.po
View File

File diff suppressed because it is too large Load Diff

1083
src/calibre/translations/ast.po
View File

File diff suppressed because it is too large Load Diff

1078
src/calibre/translations/bg.po
View File

File diff suppressed because it is too large Load Diff

1083
src/calibre/translations/bn.po
View File

File diff suppressed because it is too large Load Diff

1072
src/calibre/translations/bs.po
View File

File diff suppressed because it is too large Load Diff

1101
src/calibre/translations/ca.po
View File

File diff suppressed because it is too large Load Diff

779
src/calibre/translations/calibre.pot
View File

File diff suppressed because it is too large Load Diff

1080
src/calibre/translations/cs.po
View File

File diff suppressed because it is too large Load Diff

1075
src/calibre/translations/da.po
View File

File diff suppressed because it is too large Load Diff

1080
src/calibre/translations/de.po
View File

File diff suppressed because it is too large Load Diff

1080
src/calibre/translations/el.po
View File

File diff suppressed because it is too large Load Diff

1083
src/calibre/translations/en_AU.po
View File

File diff suppressed because it is too large Load Diff

1086
src/calibre/translations/en_CA.po
View File

File diff suppressed because it is too large Load Diff

1112
src/calibre/translations/en_GB.po
View File

File diff suppressed because it is too large Load Diff

1086
src/calibre/translations/eo.po
View File

File diff suppressed because it is too large Load Diff

1114
src/calibre/translations/es.po
View File

File diff suppressed because it is too large Load Diff

1077
src/calibre/translations/eu.po
View File

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More