Cutlery/calibre
1
0
Fork 0
mirror of https://github.com/kovidgoyal/calibre.git synced 2025-08-30 23:00:21 -04:00
Code Issues Packages Projects Releases Wiki Activity

0.7.33

Browse Source
This commit is contained in:
GRiker 2010-12-11 15:15:23 -07:00
commit b3d8fcffac
193 changed files with 246317 additions and 129300 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

176
Changelog.yaml
View File

@ -4,6 +4,182 @@
# for important features/bug fixes.
# Also, each release can have new and improved recipes.
- version: 0.7.33
date: 2010-12-10
new features:
- title: "Language sensitive sorting"
type: major
description: >
"calibre now sorts using language specific rules. The language used is the language of the calibre interface, which can be changed via Preferences->Look & Feel. There is also
a tweak that allows you to use a different language from the one used for the calibre interface. Powered by the ICU library."
- title: "Add an action to merge only formats and leave metadata alone (Shift+Alt+M)"
tickets: [7709]
- title: "Add a tweak to control which custom columns are displayed in the Book details panel."
- title: "Implement a more sophisticated 'functional programming' template language. See the User Manual for details."
- title: "Speed up deleting of large numbers of books and show progress while doing so"
- title: "Adding books: Dont refresh the Tag Browser while adding multiple books. Should speed up the adding of large numbers of books."
- title: "Edit metadata dialog: When trying to download metadata, if there are multiple matches indicate which matches have a cover and summary in the list. Also add an option to automatically download the cover of the selected match."
- title: "Drivers for the HTC Legend and Samsung Epic"
- title: "FB2 Output: Convert SVG images in the input document to raster images"
- title: "News download: Localize the navigation bars in the downloaded news to the language the user has selected for their calibre interface"
bug fixes:
- title: "Various fixes to the Title Case function"
tickets: [7846]
- title: "Content server: Fix --url-prefix being ignored for links at the Top level"
- title: "News download: When generating periodicals for the SONY use the local timezone in the SONY specific metadata"
- title: "Fix bug in cover cache that could cause it to keep a large number of covers in memory. Showed up when adding large numbers of books to calibre."
tickets: [7813]
- title: "Adding books: Run in the main thread to prevent unfortunate interactions with the metadata backup. Also fix regression that broke the Abort button."
- title: "Fix a crash on OS X if OK is clicked inthe edit metadata button while downloading a cover"
tickets: [7716]
- title: "E-book viewer: Fix a regression that prevented booksmarks from working with some EPUB files"
tickets: [7812]
- title: "Save to disk: Refactor to not open a database connection in the worker process. Also fix a bug that could lead to save failures not being reported."
- title: "Fix regression in 0.7.32 that broke opening formats in the ebook viewer from the edit metadata dialog"
- title: "FB2 Output: Generate output 100% compliant with the FB2 spec"
- title: "Fix Saved search dropdown box looses selected search"
tickets: [7787]
- title: "TXT Output: Fix an issue where the br to space conversion was not being handled properly."
improved recipes:
- Le Monde
- Ming Pao
- New Yorker
new recipes:
- title: "ToyoKeizai News and Nikkei Social News"
author: "Hiroshi Miura"
- title: "St. Louis Post Dispatch"
author: "cisaak"
- title: "Heise Open and Technology Review"
author: "Anton Gillert"
- version: 0.7.32
date: 2010-12-03
new features:
- title: "All new linux binary build. With updated libraries and replacing cx_Freeze with my own C python launcher code."
- title: "Edit metadata dialog: Add Next and Previous buttons and show cover size in tooltip"
tickets: [7706, 7711]
- title: "A new custom column type: Enumeration. This column can take one of a user defined set of values."
- title: "PML Output: Add option to reduce image sizes/bit depth to allow PML Output to be used with DropBook"
- title: "TXT Output: Add option to generate Markdown output. Turn <br> tags into spaces."
- title: "Add a count function to the template language. Make author_sort searchable."
- title: "Various consistency and usability enhancements to the search box."
tickets: [7726]
description: >
"Always select first book in result set of a search. Similar books searches added to search history. Search history order is no longer randomized. When focussing the search box with a keyboard shortcut, select all text. If you press enter in the search box, the search is executed and the book list os automatically focussed."
- title: "Driver for samsung fascinate and PocketBook 902"
- title: "FB2 Output: Add option to create FB2 sections based on internal file structure of input file (useful for EPUB files that have been split on chapter boundaries). Also add options to mark h1/h2/h3 tags as section titles in the FB2 file."
tickets: [7738]
- title: "Metadata jacket: Add publisher information to jacket."
- title: "Catalog generation: Allow use of custom columns as well as tags to indicate read books. Note that your previously saved read books setting will be lost."
- title: "Bulk metadata edit dialog: Add an Apply button to allow you to perform multiple operations in sequence"
- title: "Allow drag and drop of books onto user categories. If you drag a book from a particular column (say authors) and drop it onto a user category, the column value will be added to the user category. So for authors, the authros will be added to the user category."
- title: "Check Library can now check and repair the has_cover cache"
- title: "Allow GUI plugins to be distributed in ZIP files. See http://www.mobileread.com/forums/showthread.php?t=108774"
- title: "Allow searching by the number of tags/authors/formats/etc. See User Manual for details."
- title: "Tiny speed up when loading large libraries and make various metadata editing tasks a little faster by reducing the number of times the Tag Browser is refreshed"
bug fixes:
- title: "E-book viewer: Fix broken backwards searching"
- title: "Fix custom ratings column values being displayed incorrectly in book details area"
tickets: [7740]
- title: "Fix book details dialog not using internal viewer to view ebooks"
tickets: [7424]
- title: "MOBI Output: When the input document does not explicitly specify a size for images, set the size to be the natural size of the image. This works around Amazon's *truly wonderful* MOBI renderer's tendency to expand images that do not have a width and height specified."
- title: "Conversion pipeline: Fix bug that caused height/width specified in %/em of screen size to be incorrectly calculated by a factor of 72./DPI"
- title: "Conversion pipeline: Respect max-width and max-height when calculating the effective size of an element"
- title: "Conversion pipeline: Do not override CSS for images with the value of the img width/height attributes, unless no CSS is specified for the image"
- title: "E-book viewer: Resize automatically to fit on smaller screens"
- title: "Use the same MIME database on all platforms that calibre runs on, works around python 2.7's crazy insistence on reading MIME data from the registry"
- title: "Kobo driver: Allow html, txt and rtf documents to be deleted"
- title: "Always overwrite title/author metadata when downloading metadata for books added by ISBN"
- title: "Nook Color profile: Reduce screen height to 900px"
- title: "Fix regression that broke RTF conversion on some linux systems"
- title: "Fix bug that could break searching after copying and deleting a book from the current library"
tickets: [7459]
improved recipes:
- NZZ
- Frankfurter Rundschau
- JiJi Press
- Revista Muy Intersante
new recipes:
- title: "Global Times"
author: "malfi"
- title: "The Philosopher's Magazine"
author: "Darko Miletic"
- title: "Poughkeepsie Journal"
author: "weebl"
- title: "Business Spectator and ABC Australia"
author: "Dean Cording"
- title: "La Rijoa and NacionRed"
author: "Arturo Martinez Nieves"
- title: "Animal Politico"
author: "leamsi"
- version: 0.7.31
date: 2010-11-27

26
resources/default_tweaks.py
View File

@ -181,19 +181,25 @@ max_content_server_tags_shown=5
# content_server_will_display is a list of custom fields to be displayed.
# content_server_wont_display is a list of custom fields not to be displayed.
# wont_display has priority over will_display.
# The special value '*' means all custom fields.
# The special value '*' means all custom fields. The value [] means no entries.
# Defaults:
# content_server_will_display = ['*']
# content_server_wont_display = ['']
# content_server_wont_display = []
# Examples:
# To display only the custom fields #mytags and #genre:
# content_server_will_display = ['#mytags', '#genre']
# content_server_wont_display = ['']
# content_server_wont_display = []
# To display all fields except #mycomments:
# content_server_will_display = ['*']
# content_server_wont_display['#mycomments']
content_server_will_display = ['*']
content_server_wont_display = ['']
content_server_wont_display = []
# Same as above (content server) but for the book details pane. Same syntax.
# As above, this tweak affects only display of custom fields. The standard
# fields are not affected
book_details_will_display = ['*']
book_details_wont_display = []
# Set the maximum number of sort 'levels' that calibre will use to resort the
@ -217,3 +223,15 @@ generate_cover_foot_font = None
# open_viewer, do_nothing, edit_cell. Default: open_viewer.
# Example: doubleclick_on_library_view = 'do_nothing'
doubleclick_on_library_view = 'open_viewer'
# Language to use when sorting. Setting this tweak will force sorting to use the
# collating order for the specified language. This might be useful if you run
# calibre in English but want sorting to work in the language where you live.
# Set the tweak to the desired ISO 639-1 language code, in lower case.
# You can find the list of supported locales at
# http://publib.boulder.ibm.com/infocenter/iseries/v5r3/topic/nls/rbagsicusortsequencetables.htm
# Default: locale_for_sorting = '' -- use the language calibre displays in
# Example: locale_for_sorting = 'fr' -- sort using French rules.
# Example: locale_for_sorting = 'nb' -- sort using Norwegian rules.
locale_for_sorting = ''

BIN
resources/images/news/tpm_uk.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 873 B

46
resources/recipes/globaltimes.recipe Normal file
View File

@ -0,0 +1,46 @@
from calibre.web.feeds.news import BasicNewsRecipe
import re
class globaltimes(BasicNewsRecipe):
title = u'Global Times'
__author__ = 'malfi'
language = 'zh'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
cover_url = 'http://enhimg2.huanqiu.com/images/logo.png'
language = 'en'
keep_only_tags = []
keep_only_tags.append(dict(name = 'div', attrs = {'id': 'content'}))
remove_tags = []
remove_tags.append(dict(name = 'div', attrs = {'class': 'location'}))
remove_tags.append(dict(name = 'div', attrs = {'class': 'contentpage'}))
remove_tags.append(dict(name = 'li', attrs = {'id': 'pl'}))
extra_css = '''
h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
'''
def parse_index(self):
catnames = {}
catnames["http://china.globaltimes.cn/chinanews/"] = "China Politics"
catnames["http://china.globaltimes.cn/diplomacy/"] = "China Diplomacy"
catnames["http://military.globaltimes.cn/china/"] = "China Military"
catnames["http://business.globaltimes.cn/china-economy/"] = "China Economy"
catnames["http://world.globaltimes.cn/asia-pacific/"] = "Asia Pacific"
feeds = []
for cat in catnames.keys():
articles = []
soup = self.index_to_soup(cat)
for a in soup.findAll('a',attrs={'href' : re.compile(cat+"201[0-9]-[0-1][0-9]/[0-9][0-9][0-9][0-9][0-9][0-9].html")}):
url = a['href'].strip()
myarticle=({'title':self.tag_to_string(a), 'url':url, 'description':'', 'date':''})
self.log("found %s" % url)
articles.append(myarticle)
self.log("Adding URL %s\n" %url)
if articles:
feeds.append((catnames[cat], articles))
return feeds

38
resources/recipes/heise_open.recipe Normal file
View File

@ -0,0 +1,38 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Anton Gillert <atx at binaryninja.de>'
'''
Fetch Heise Open.
'''
from calibre.web.feeds.news import BasicNewsRecipe
class HeiseOpenDe(BasicNewsRecipe):
title = 'Heise Open'
description = 'Opensource news from Germany'
__author__ = 'Anton Gillert'
use_embedded_content = False
language = 'de'
timefmt = ' [%d %b %Y]'
max_articles_per_feed = 40
no_stylesheets = True
feeds = [ ('Heise Open', 'http://www.heise.de/open/news/news-atom.xml') ]
def print_version(self, url):
return url + '?view=print'
remove_tags = [dict(id='navi_top'),
dict(id='navi_bottom'),
dict(name='div', attrs={'class':'navi_top_logo'}),
dict(name='img', attrs={'src':'/open/icons/open_logo_2009_weiss.gif'}),
dict(name='h5', attrs={'style':'margin: 0.5em 0;'}),
dict(name='p', attrs={'class':'news_datum'}),
dict(name='p', attrs={'class':'size80'})]
remove_tags_after = [dict(name='p', attrs={'class':'size80'})]
def get_cover_url(self):
return 'http://www.heise.de/open/icons/open_logo_2009_weiss.gif'

36
resources/recipes/kompiutierra.recipe Normal file
View File

@ -0,0 +1,36 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
__license__ = 'GPL v3'
__copyright__ = '2010, Vadim Dyadkin, dyadkin@gmail.com'
__author__ = 'Vadim Dyadkin'
from calibre.web.feeds.news import BasicNewsRecipe
class Computerra(BasicNewsRecipe):
title = u'\u041a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u0440\u0430'
recursion = 50
oldest_article = 100
__author__ = 'Vadim Dyadkin'
max_articles_per_feed = 100
use_embedded_content = False
simultaneous_downloads = 5
language = 'ru'
description = u'\u041a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u044b, \u043e\u043a\u043e\u043b\u043e\u043d\u0430\u0443\u0447\u043d\u044b\u0435 \u0438 \u043e\u043a\u043e\u043b\u043e\u0444\u0438\u043b\u043e\u0441\u043e\u0444\u0441\u043a\u0438\u0435 \u0441\u0442\u0430\u0442\u044c\u0438, \u0433\u0430\u0434\u0436\u0435\u0442\u044b.'
keep_only_tags = [dict(name='div', attrs={'id': 'content'}),]
feeds = [(u'\u041a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u0440\u0430', 'http://feeds.feedburner.com/ct_news/'),]
remove_tags = [dict(name='div', attrs={'id': ['fin', 'idc-container', 'idc-noscript',]}),
dict(name='ul', attrs={'class': "related_post"}),
dict(name='p', attrs={'class': 'info'}),
dict(name='a', attrs={'rel': 'tag', 'class': 'twitter-share-button', 'type': 'button_count'}),
dict(name='h2', attrs={}),]
extra_css = 'body { text-align: justify; }'
def get_article_url(self, article):
return article.get('feedburner:origLink', article.get('guid'))

206
resources/recipes/le_monde.recipe
View File

@ -1,106 +1,89 @@
#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2008, Mathieu Godlewski <mathieu at godlewski.fr>'
'''
lemonde.fr
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.web.feeds.recipes import BasicNewsRecipe
class LeMonde(BasicNewsRecipe):
title = 'LeMonde.fr'
__author__ = 'Mathieu Godlewski and Sujata Raman'
description = 'Global news in french'
oldest_article = 3
language = 'fr'
title = 'Le Monde'
__author__ = 'veezh'
description = 'Actualités'
oldest_article = 1
max_articles_per_feed = 100
no_stylesheets = True
#delay = 1
use_embedded_content = False
encoding = 'cp1252'
publisher = 'lemonde.fr'
language = 'fr'
conversion_options = {
'comments' : description
,'language' : language
,'publisher' : publisher
,'linearize_tables': True
}
max_articles_per_feed = 30
no_stylesheets = True
remove_javascript = True
remove_empty_feeds = True
filterDuplicates = True
# cover_url='http://abonnes.lemonde.fr/titresdumonde/'+date.today().strftime("%y%m%d")+'/1.jpg'
def preprocess_html(self, soup):
for alink in soup.findAll('a'):
if alink.string is not None:
tstr = alink.string
alink.replaceWith(tstr)
return soup
extra_css = '''
.dateline{color:#666666;font-family:verdana,sans-serif;font-size:x-small;}
.author{font-family:verdana,sans-serif;font-size:x-small;color:#222222;}
.articleImage{color:#666666;font-family:verdana,sans-serif;font-size:x-small;}
.mainText{font-family:Georgia,serif;color:#222222;}
.LM_articleText{font-family:Arial,Helvetica,sans-serif;}
.LM_titleZone{font-family:Arial,Helvetica,sans-serif;}
.mainContent{font-family:Georgia,serif;}
.LM_content{font-family:Georgia,serif;}
.LM_caption{font-family:Georgia,serif;font-size:-small;}
.LM_imageSource{font-family:Arial,Helvetica,sans-serif;font-size:x-small;color:#666666;}
h1{font-family:Arial,Helvetica,sans-serif;font-size:medium;color:#000000;}
.post{font-family:Arial,Helvetica,sans-serif;}
.mainTitle{font-family:Georgia,serif;}
.content{font-family:Georgia,serif;}
.entry{font-family:Georgia,serif;}
h2{font-family:Arial,Helvetica,sans-serif;font-size:large;}
small{font-family:Arial,Helvetica,sans-serif; color:#ED1B23;}
'''
feeds = [
('A la Une', 'http://www.lemonde.fr/rss/une.xml'),
('International', 'http://www.lemonde.fr/rss/sequence/0,2-3210,1-0,0.xml'),
('Europe', 'http://www.lemonde.fr/rss/sequence/0,2-3214,1-0,0.xml'),
('Societe', 'http://www.lemonde.fr/rss/sequence/0,2-3224,1-0,0.xml'),
('Economie', 'http://www.lemonde.fr/rss/sequence/0,2-3234,1-0,0.xml'),
('Medias', 'http://www.lemonde.fr/rss/sequence/0,2-3236,1-0,0.xml'),
('Rendez-vous', 'http://www.lemonde.fr/rss/sequence/0,2-3238,1-0,0.xml'),
('Sports', 'http://www.lemonde.fr/rss/sequence/0,2-3242,1-0,0.xml'),
('Planete', 'http://www.lemonde.fr/rss/sequence/0,2-3244,1-0,0.xml'),
('Culture', 'http://www.lemonde.fr/rss/sequence/0,2-3246,1-0,0.xml'),
('Technologies', 'http://www.lemonde.fr/rss/sequence/0,2-651865,1-0,0.xml'),
('Cinema', 'http://www.lemonde.fr/rss/sequence/0,2-3476,1-0,0.xml'),
('Voyages', 'http://www.lemonde.fr/rss/sequence/0,2-3546,1-0,0.xml'),
('Livres', 'http://www.lemonde.fr/rss/sequence/0,2-3260,1-0,0.xml'),
('Examens', 'http://www.lemonde.fr/rss/sequence/0,2-3404,1-0,0.xml'),
('Opinions', 'http://www.lemonde.fr/rss/sequence/0,2-3232,1-0,0.xml')
]
keep_only_tags = [dict(name='div', attrs={'id':["mainTitle","mainContent","LM_content","content"]}),
dict(name='div', attrs={'class':["post"]})
]
remove_tags = [dict(name='img', attrs={'src':'http://medias.lemonde.fr/mmpub/img/lgo/lemondefr_pet.gif'}),
dict(name='div', attrs={'id':'xiti-logo-noscript'}),
dict(name='br', attrs={}),
dict(name='iframe', attrs={}),
dict(name='table', attrs={'id':["toolBox"]}),
dict(name='table', attrs={'class':["bottomToolBox"]}),
dict(name='div', attrs={'class':["pageNavigation","LM_pagination","fenetreBoxesContainer","breakingNews","LM_toolsBottom","LM_comments","LM_tools","pave_meme_sujet_hidden","boxMemeSujet"]}),
dict(name='div', attrs={'id':["miniUne","LM_sideBar"]}),
]
preprocess_regexps = [ (re.compile(i[0], re.IGNORECASE|re.DOTALL), i[1]) for i in
[
(r'<html.*(<div class="post".*?>.*?</div>.*?<div class="entry">.*?</div>).*You can start editing here.*</html>', lambda match : '<html><body>'+match.group(1)+'</body></html>'),
(r'<p>&nbsp;</p>', lambda match : ''),
(r'<img src="http://medias\.lemonde\.fr/mmpub/img/let/(.)\.gif"[^>]*><div class=ar-txt>', lambda match : '<div class=ar-txt>'+match.group(1).upper()),
(r'<img src="http://medias\.lemonde\.fr/mmpub/img/let/q(.)\.gif"[^>]*><div class=ar-txt>', lambda match : '<div class=ar-txt>"'+match.group(1).upper()),
(r'(<div class=desc><b>.*</b></div>).*</body>', lambda match : match.group(1)),
preprocess_regexps = [
(re.compile(r' \''), lambda match: ' &lsquo;'),
(re.compile(r'\''), lambda match: '&rsquo;'),
(re.compile(r'"<'), lambda match: '&nbsp;&raquo;<'),
(re.compile(r'>"'), lambda match: '>&laquo;&nbsp;'),
(re.compile(r'&rsquo;"'), lambda match: '&rsquo;«&nbsp;'),
(re.compile(r' "'), lambda match: ' &laquo;&nbsp;'),
(re.compile(r'" '), lambda match: '&nbsp;&raquo; '),
(re.compile(r'\("'), lambda match: '(&laquo;&nbsp;'),
(re.compile(r'"\)'), lambda match: '&nbsp;&raquo;)'),
(re.compile(r'"\.'), lambda match: '&nbsp;&raquo;.'),
(re.compile(r'",'), lambda match: '&nbsp;&raquo;,'),
(re.compile(r'"\?'), lambda match: '&nbsp;&raquo;?'),
(re.compile(r'":'), lambda match: '&nbsp;&raquo;:'),
(re.compile(r'";'), lambda match: '&nbsp;&raquo;;'),
(re.compile(r'"\!'), lambda match: '&nbsp;&raquo;!'),
(re.compile(r' :'), lambda match: '&nbsp;:'),
(re.compile(r' ;'), lambda match: '&nbsp;;'),
(re.compile(r' \?'), lambda match: '&nbsp;?'),
(re.compile(r' \!'), lambda match: '&nbsp;!'),
(re.compile(r'\s»'), lambda match: '&nbsp;»'),
(re.compile(r'«\s'), lambda match: '«&nbsp;'),
(re.compile(r' %'), lambda match: '&nbsp;%'),
(re.compile(r'\.jpg&nbsp;&raquo; border='), lambda match: '.jpg'),
(re.compile(r'\.png&nbsp;&raquo; border='), lambda match: '.png'),
]
]
article_match_regexps = [ (re.compile(i)) for i in
[
(r'http://www\.lemonde\.fr/\S+/article/.*'),
(r'http://www\.lemonde\.fr/\S+/portfolio/.*'),
(r'http://www\.lemonde\.fr/\S+/article_interactif/.*'),
(r'http://\S+\.blog\.lemonde\.fr/.*'),
]
]
keep_only_tags = [
dict(name='div', attrs={'class':['contenu']})
]
# def print_version(self, url):
# return re.sub('http://www\.lemonde\.fr/.*_([0-9]+)_[0-9]+\.html.*','http://www.lemonde.fr/web/imprimer_element/0,40-0,50-\\1,0.html' ,url)
remove_tags_after = [dict(id='appel_temoignage')]
# Used to filter duplicated articles
articles_list = []
def get_article_url(self, article):
link = article.get('link')
if 'blog' not in link:
return link
feeds = [
('A la une', 'http://www.lemonde.fr/rss/une.xml'),
('International', 'http://www.lemonde.fr/rss/tag/international.xml'),
('Europe', 'http://www.lemonde.fr/rss/tag/europe.xml'),
(u'Société', 'http://www.lemonde.fr/rss/tag/societe.xml'),
('Economie', 'http://www.lemonde.fr/rss/tag/economie.xml'),
(u'Médias', 'http://www.lemonde.fr/rss/tag/actualite-medias.xml'),
(u'Planète', 'http://www.lemonde.fr/rss/tag/planete.xml'),
('Culture', 'http://www.lemonde.fr/rss/tag/culture.xml'),
('Technologies', 'http://www.lemonde.fr/rss/tag/technologies.xml'),
('Livres', 'http://www.lemonde.fr/rss/tag/livres.xml'),
]
def get_cover_url(self):
cover_url = None
@ -111,42 +94,3 @@ class LeMonde(BasicNewsRecipe):
cover_url = link_item.img['src']
return cover_url
def get_article_url(self, article):
url=article.get('link', None)
url=url[0:url.find("#")]
if url in self.articles_list:
self.log_debug(_('Skipping duplicated article: %s')%url)
return False
if self.is_article_wanted(url):
self.articles_list.append(url)
if '/portfolio/' in url or '/video/' in url:
url = None
return url
self.log_debug(_('Skipping filtered article: %s')%url)
url = article.get('guid', None)
return False
def is_article_wanted(self, url):
if self.article_match_regexps:
for m in self.article_match_regexps:
if m.search(url):
return True
return False
return False
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll(face=True):
del item['face']
for tag in soup.findAll(name=['ul','li']):
tag.name = 'div'
return soup

16
resources/recipes/mainichi.recipe
View File

@ -4,6 +4,7 @@ __copyright__ = '2010, Hiroshi Miura <miurahr@linux.com>'
www.mainichi.jp
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe
class MainichiDailyNews(BasicNewsRecipe):
@ -22,3 +23,18 @@ class MainichiDailyNews(BasicNewsRecipe):
remove_tags = [{'class':"RelatedArticle"}]
remove_tags_after = {'class':"Credit"}
def parse_feeds(self):
feeds = BasicNewsRecipe.parse_feeds(self)
for curfeed in feeds:
delList = []
for a,curarticle in enumerate(curfeed.articles):
if re.search(r'pheedo.jp', curarticle.url):
delList.append(curarticle)
if len(delList)>0:
for d in delList:
index = curfeed.articles.index(d)
curfeed.articles[index:index+1] = []
return feeds

16
resources/recipes/mainichi_it_news.recipe
View File

@ -1,4 +1,5 @@
from calibre.web.feeds.news import BasicNewsRecipe
import re
class MainichiDailyITNews(BasicNewsRecipe):
title = u'\u6bce\u65e5\u65b0\u805e(IT&\u5bb6\u96fb)'
@ -16,3 +17,18 @@ class MainichiDailyITNews(BasicNewsRecipe):
remove_tags = [{'class':"RelatedArticle"}]
remove_tags_after = {'class':"Credit"}
def parse_feeds(self):
feeds = BasicNewsRecipe.parse_feeds(self)
for curfeed in feeds:
delList = []
for a,curarticle in enumerate(curfeed.articles):
if re.search(r'pheedo.jp', curarticle.url):
delList.append(curarticle)
if len(delList)>0:
for d in delList:
index = curfeed.articles.index(d)
curfeed.articles[index:index+1] = []
return feeds

151
resources/recipes/ming_pao.recipe
View File

@ -1,8 +1,9 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Eddie Lau'
'''
modified from Singtao Toronto calibre recipe by rty
Change Log:
2010/12/07: add entertainment section, use newspaper front page as ebook cover, suppress date display in section list
(to avoid wrong date display in case the user generates the ebook in a time zone different from HKT)
2010/11/22: add English section, remove eco-news section which is not updated daily, correct
ordering of articles
2010/11/12: add news image and eco-news section
@ -17,14 +18,15 @@ from calibre.web.feeds.recipes import BasicNewsRecipe
from contextlib import nested
from calibre import __appname__, strftime
from calibre import __appname__
from calibre.ebooks.BeautifulSoup import BeautifulSoup
from calibre.ebooks.metadata.opf2 import OPFCreator
from calibre.ebooks.metadata.toc import TOC
from calibre.ebooks.metadata import MetaInformation
from calibre.utils.date import now as nowf
class MPHKRecipe(BasicNewsRecipe):
IsKindleUsed = True # to avoid generating periodical in which CJK characters can't be displayed in section/article view
title = 'Ming Pao - Hong Kong'
oldest_article = 1
max_articles_per_feed = 100
@ -39,13 +41,13 @@ class MPHKRecipe(BasicNewsRecipe):
encoding = 'Big5-HKSCS'
recursions = 0
conversion_options = {'linearize_tables':True}
extra_css = 'img {display: block; margin-left: auto; margin-right: auto; margin-top: 10px; margin-bottom: 10px;}'
#extra_css = 'img {float:right; margin:4px;}'
timefmt = ''
extra_css = 'img {display: block; margin-left: auto; margin-right: auto; margin-top: 10px; margin-bottom: 10px;} font>b {font-size:200%; font-weight:bold;}'
masthead_url = 'http://news.mingpao.com/image/portals_top_logo_news.gif'
keep_only_tags = [dict(name='h1'),
#dict(name='font', attrs={'style':['font-size:14pt; line-height:160%;']}), # for entertainment page
dict(name='font', attrs={'style':['font-size:14pt; line-height:160%;']}), # for entertainment page title
dict(attrs={'class':['photo']}),
dict(attrs={'id':['newscontent']}),
dict(attrs={'id':['newscontent']}), # entertainment page content
dict(attrs={'id':['newscontent01','newscontent02']})]
remove_tags = [dict(name='style'),
dict(attrs={'id':['newscontent135']})] # for the finance page
@ -55,51 +57,68 @@ class MPHKRecipe(BasicNewsRecipe):
lambda match: '<h1>'),
(re.compile(r'</h5>', re.DOTALL|re.IGNORECASE),
lambda match: '</h1>'),
(re.compile(r'<p><a href=.+?</a></p>', re.DOTALL|re.IGNORECASE), # for entertainment page
lambda match: '')
]
def image_url_processor(cls, baseurl, url):
# trick: break the url at the first occurance of digit, add an additional
# '_' at the front
# not working, may need to move this to preprocess_html() method
#minIdx = 10000
#i0 = url.find('0')
#if i0 >= 0 and i0 < minIdx:
# minIdx = i0
#i1 = url.find('1')
#if i1 >= 0 and i1 < minIdx:
# minIdx = i1
#i2 = url.find('2')
#if i2 >= 0 and i2 < minIdx:
# minIdx = i2
#i3 = url.find('3')
#if i3 >= 0 and i0 < minIdx:
# minIdx = i3
#i4 = url.find('4')
#if i4 >= 0 and i4 < minIdx:
# minIdx = i4
#i5 = url.find('5')
#if i5 >= 0 and i5 < minIdx:
# minIdx = i5
#i6 = url.find('6')
#if i6 >= 0 and i6 < minIdx:
# minIdx = i6
#i7 = url.find('7')
#if i7 >= 0 and i7 < minIdx:
# minIdx = i7
#i8 = url.find('8')
#if i8 >= 0 and i8 < minIdx:
# minIdx = i8
#i9 = url.find('9')
#if i9 >= 0 and i9 < minIdx:
# minIdx = i9
#return url[0:minIdx] + '_' + url[minIdx+1:]
# minIdx = 10000
# i0 = url.find('0')
# if i0 >= 0 and i0 < minIdx:
# minIdx = i0
# i1 = url.find('1')
# if i1 >= 0 and i1 < minIdx:
# minIdx = i1
# i2 = url.find('2')
# if i2 >= 0 and i2 < minIdx:
# minIdx = i2
# i3 = url.find('3')
# if i3 >= 0 and i0 < minIdx:
# minIdx = i3
# i4 = url.find('4')
# if i4 >= 0 and i4 < minIdx:
# minIdx = i4
# i5 = url.find('5')
# if i5 >= 0 and i5 < minIdx:
# minIdx = i5
# i6 = url.find('6')
# if i6 >= 0 and i6 < minIdx:
# minIdx = i6
# i7 = url.find('7')
# if i7 >= 0 and i7 < minIdx:
# minIdx = i7
# i8 = url.find('8')
# if i8 >= 0 and i8 < minIdx:
# minIdx = i8
# i9 = url.find('9')
# if i9 >= 0 and i9 < minIdx:
# minIdx = i9
return url
def get_fetchdate(self):
def get_dtlocal(self):
dt_utc = datetime.datetime.utcnow()
# convert UTC to local hk time - at around HKT 6.00am, all news are available
dt_local = dt_utc - datetime.timedelta(-2.0/24)
return dt_local.strftime("%Y%m%d")
return dt_local
def get_fetchdate(self):
return self.get_dtlocal().strftime("%Y%m%d")
def get_fetchday(self):
# convert UTC to local hk time - at around HKT 6.00am, all news are available
return self.get_dtlocal().strftime("%d")
def get_cover_url(self):
cover = 'http://news.mingpao.com/' + self.get_fetchdate() + '/' + self.get_fetchdate() + '_' + self.get_fetchday() + 'gacov.jpg'
br = BasicNewsRecipe.get_browser()
try:
br.open(cover)
except:
cover = None
return cover
def parse_index(self):
feeds = []
@ -127,9 +146,9 @@ class MPHKRecipe(BasicNewsRecipe):
# if eco_articles:
# feeds.append((u'\u74b0\u4fdd Eco News', eco_articles))
# special - entertainment
#ent_articles = self.parse_ent_section('http://ol.mingpao.com/cfm/star1.cfm')
#if ent_articles:
# feeds.append(('Entertainment', ent_articles))
ent_articles = self.parse_ent_section('http://ol.mingpao.com/cfm/star1.cfm')
if ent_articles:
feeds.append((u'\u5f71\u8996 Entertainment', ent_articles))
return feeds
def parse_section(self, url):
@ -164,6 +183,7 @@ class MPHKRecipe(BasicNewsRecipe):
return current_articles
def parse_eco_section(self, url):
dateStr = self.get_fetchdate()
soup = self.index_to_soup(url)
divs = soup.findAll(attrs={'class': ['bullet']})
current_articles = []
@ -173,23 +193,25 @@ class MPHKRecipe(BasicNewsRecipe):
title = self.tag_to_string(a)
url = a.get('href', False)
url = 'http://tssl.mingpao.com/htm/marketing/eco/cfm/' +url
if url not in included_urls and url.rfind('Redirect') == -1:
if url not in included_urls and url.rfind('Redirect') == -1 and not url.rfind('.txt') == -1 and not url.rfind(dateStr) == -1:
current_articles.append({'title': title, 'url': url, 'description':''})
included_urls.append(url)
return current_articles
#def parse_ent_section(self, url):
# dateStr = self.get_fetchdate()
# soup = self.index_to_soup(url)
# a = soup.findAll('a', href=True)
# current_articles = []
# included_urls = []
# for i in a:
# title = self.tag_to_string(i)
# url = 'http://ol.mingpao.com/cfm/' + i.get('href', False)
# if url not in included_urls and not url.rfind('.txt') == -1 and not url.rfind(dateStr) == -1 and not title == '':
# current_articles.append({'title': title, 'url': url, 'description': ''})
# return current_articles
def parse_ent_section(self, url):
soup = self.index_to_soup(url)
a = soup.findAll('a', href=True)
a.reverse()
current_articles = []
included_urls = []
for i in a:
title = self.tag_to_string(i)
url = 'http://ol.mingpao.com/cfm/' + i.get('href', False)
if (url not in included_urls) and (not url.rfind('.txt') == -1) and (not url.rfind('star') == -1):
current_articles.append({'title': title, 'url': url, 'description': ''})
included_urls.append(url)
current_articles.reverse()
return current_articles
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
@ -201,21 +223,26 @@ class MPHKRecipe(BasicNewsRecipe):
return soup
def create_opf(self, feeds, dir=None):
#super(MPHKRecipe,self).create_opf(feeds, dir)
if self.IsKindleUsed == False:
super(MPHKRecipe,self).create_opf(feeds, dir)
return
if dir is None:
dir = self.output_dir
title = self.short_title()
if self.output_profile.periodical_date_in_title:
title += strftime(self.timefmt)
title += ' ' + self.get_fetchdate()
#if self.output_profile.periodical_date_in_title:
# title += strftime(self.timefmt)
mi = MetaInformation(title, [__appname__])
mi.publisher = __appname__
mi.author_sort = __appname__
mi.publication_type = self.publication_type+':'+self.short_title()
mi.timestamp = nowf()
#mi.timestamp = nowf()
mi.timestamp = self.get_dtlocal()
mi.comments = self.description
if not isinstance(mi.comments, unicode):
mi.comments = mi.comments.decode('utf-8', 'replace')
mi.pubdate = nowf()
#mi.pubdate = nowf()
mi.pubdate = self.get_dtlocal()
opf_path = os.path.join(dir, 'index.opf')
ncx_path = os.path.join(dir, 'index.ncx')
opf = OPFCreator(dir, mi)

27
resources/recipes/new_yorker.recipe
View File

@ -22,8 +22,19 @@ class NewYorker(BasicNewsRecipe):
masthead_url = 'http://www.newyorker.com/css/i/hed/logo.gif'
extra_css = """
body {font-family: "Times New Roman",Times,serif}
.articleauthor{color: #9F9F9F; font-family: Arial, sans-serif; font-size: small; text-transform: uppercase}
.rubric{color: #CD0021; font-family: Arial, sans-serif; font-size: small; text-transform: uppercase}
.articleauthor{color: #9F9F9F;
font-family: Arial, sans-serif;
font-size: small;
text-transform: uppercase}
.rubric,.dd,h6#credit{color: #CD0021;
font-family: Arial, sans-serif;
font-size: small;
text-transform: uppercase}
.descender:first-letter{display: inline; font-size: xx-large; font-weight: bold}
.dd,h6#credit{color: gray}
.c{display: block}
.caption,h2#articleintro{font-style: italic}
.caption{font-size: small}
"""
conversion_options = {
@ -39,7 +50,7 @@ class NewYorker(BasicNewsRecipe):
]
remove_tags = [
dict(name=['meta','iframe','base','link','embed','object'])
,dict(attrs={'class':['utils','articleRailLinks','icons'] })
,dict(attrs={'class':['utils','socialUtils','articleRailLinks','icons'] })
,dict(attrs={'id':['show-header','show-footer'] })
]
remove_attributes = ['lang']
@ -59,3 +70,13 @@ class NewYorker(BasicNewsRecipe):
cover_url = 'http://www.newyorker.com' + cover_item['src'].strip()
return cover_url
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
auth = soup.find(attrs={'id':'articleauthor'})
if auth:
alink = auth.find('a')
if alink and alink.string is not None:
txt = alink.string
alink.replaceWith(txt)
return soup

5
resources/recipes/nikkei_sub_life.recipe
View File

@ -32,12 +32,9 @@ class NikkeiNet_sub_life(BasicNewsRecipe):
remove_tags_after = {'class':"cmn-pr_list"}
feeds = [ (u'\u304f\u3089\u3057', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=kurashi'),
(u'\u30b9\u30dd\u30fc\u30c4', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=sports'),
(u'\u793e\u4f1a', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=shakai'),
(u'\u30a8\u30b3', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=eco'),
(u'\u5065\u5eb7', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=kenkou'),
(u'\u7279\u96c6', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=special'),
(u'\u30e9\u30f3\u30ad\u30f3\u30b0', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=ranking')
(u'\u7279\u96c6', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=special')
]
def get_browser(self):

102
resources/recipes/nikkei_sub_shakai.recipe Normal file
View File

@ -0,0 +1,102 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Hiroshi Miura <miurahr@linux.com>'
'''
www.nikkei.com
'''
import re
from calibre.web.feeds.recipes import BasicNewsRecipe
import mechanize
from calibre.ptempfile import PersistentTemporaryFile
class NikkeiNet_sub_life(BasicNewsRecipe):
title = u'\u65e5\u7d4c\u65b0\u805e\u96fb\u5b50\u7248(\u751f\u6d3b)'
__author__ = 'Hiroshi Miura'
description = 'News and current market affairs from Japan'
cover_url = 'http://parts.nikkei.com/parts/ds/images/common/logo_r1.svg'
masthead_url = 'http://parts.nikkei.com/parts/ds/images/common/logo_r1.svg'
needs_subscription = True
oldest_article = 2
max_articles_per_feed = 20
language = 'ja'
remove_javascript = False
temp_files = []
remove_tags_before = {'class':"cmn-section cmn-indent"}
remove_tags = [
{'class':"JSID_basePageMove JSID_baseAsyncSubmit cmn-form_area JSID_optForm_utoken"},
{'class':"cmn-article_keyword cmn-clearfix"},
{'class':"cmn-print_headline cmn-clearfix"},
]
remove_tags_after = {'class':"cmn-pr_list"}
feeds = [
(u'\u793e\u4f1a', u'http://www.zou3.net/php/rss/nikkei2rss.php?head=shakai')
]
def get_browser(self):
br = BasicNewsRecipe.get_browser()
cj = mechanize.LWPCookieJar()
br.set_cookiejar(cj)
#br.set_debug_http(True)
#br.set_debug_redirects(True)
#br.set_debug_responses(True)
if self.username is not None and self.password is not None:
#print "----------------------------get login form--------------------------------------------"
# open login form
br.open('https://id.nikkei.com/lounge/nl/base/LA0010.seam')
response = br.response()
#print "----------------------------get login form---------------------------------------------"
#print "----------------------------set login form---------------------------------------------"
# remove disabled input which brings error on mechanize
response.set_data(response.get_data().replace("<input id=\"j_id48\"", "<!-- "))
response.set_data(response.get_data().replace("gm_home_on.gif\" />", " -->"))
br.set_response(response)
br.select_form(name='LA0010Form01')
br['LA0010Form01:LA0010Email'] = self.username
br['LA0010Form01:LA0010Password'] = self.password
br.form.find_control(id='LA0010Form01:LA0010AutoLoginOn',type="checkbox").get(nr=0).selected = True
br.submit()
br.response()
#print "----------------------------send login form---------------------------------------------"
#print "----------------------------open news main page-----------------------------------------"
# open news site
br.open('http://www.nikkei.com/')
br.response()
#print "----------------------------www.nikkei.com BODY --------------------------------------"
#print response2.get_data()
#print "-------------------------^^-got auto redirect form----^^--------------------------------"
# forced redirect in default
br.select_form(nr=0)
br.submit()
response3 = br.response()
# return some cookie which should be set by Javascript
#print response3.geturl()
raw = response3.get_data()
#print "---------------------------response to form --------------------------------------------"
# grab cookie from JS and set it
redirectflag = re.search(r"var checkValue = '(\d+)';", raw, re.M).group(1)
br.select_form(nr=0)
self.temp_files.append(PersistentTemporaryFile('_fa.html'))
self.temp_files[-1].write("#LWP-Cookies-2.0\n")
self.temp_files[-1].write("Set-Cookie3: Cookie-dummy=Cookie-value; domain=\".nikkei.com\"; path=\"/\"; path_spec; secure; expires=\"2029-12-21 05:07:59Z\"; version=0\n")
self.temp_files[-1].write("Set-Cookie3: redirectFlag="+redirectflag+"; domain=\".nikkei.com\"; path=\"/\"; path_spec; secure; expires=\"2029-12-21 05:07:59Z\"; version=0\n")
self.temp_files[-1].close()
cj.load(self.temp_files[-1].name)
br.submit()
#br.set_debug_http(False)
#br.set_debug_redirects(False)
#br.set_debug_responses(False)
return br

20
resources/recipes/nin.recipe
View File

@ -8,8 +8,8 @@ www.nin.co.rs
import re
from calibre import strftime
from calibre.web.feeds.news import BasicNewsRecipe
from contextlib import nested, closing
from calibre.ebooks.BeautifulSoup import BeautifulSoup, NavigableString, CData, Tag
from contextlib import closing
from calibre.ebooks.BeautifulSoup import BeautifulSoup
from calibre import entity_to_unicode
class Nin(BasicNewsRecipe):
@ -29,14 +29,14 @@ class Nin(BasicNewsRecipe):
use_embedded_content = False
language = 'sr'
publication_type = 'magazine'
extra_css = """
extra_css = """
@font-face {font-family: "sans1";src:url(res:///opt/sony/ebook/FONT/tt0003m_.ttf)}
body{font-family: Verdana, Lucida, sans1, sans-serif}
.article_description{font-family: Verdana, Lucida, sans1, sans-serif}
.artTitle{font-size: x-large; font-weight: bold; color: #900}
.izjava{font-size: x-large; font-weight: bold}
.columnhead{font-size: small; font-weight: bold;}
img{margin-top:0.5em; margin-bottom: 0.7em; display: block}
body{font-family: Verdana, Lucida, sans1, sans-serif}
.article_description{font-family: Verdana, Lucida, sans1, sans-serif}
.artTitle{font-size: x-large; font-weight: bold; color: #900}
.izjava{font-size: x-large; font-weight: bold}
.columnhead{font-size: small; font-weight: bold;}
img{margin-top:0.5em; margin-bottom: 0.7em; display: block}
b{margin-top: 1em}
"""
@ -148,4 +148,4 @@ class Nin(BasicNewsRecipe):
img.extract()
tbl.replaceWith(img)
return soup

6
resources/recipes/nytimes.recipe
View File

@ -282,9 +282,9 @@ class NYTimes(BasicNewsRecipe):
br = BasicNewsRecipe.get_browser()
if self.username is not None and self.password is not None:
br.open('http://www.nytimes.com/auth/login')
br.select_form(name='login')
br['USERID'] = self.username
br['PASSWORD'] = self.password
br.form = br.forms().next()
br['userid'] = self.username
br['password'] = self.password
raw = br.submit().read()
if 'Please try again' in raw:
raise Exception('Your username and password are incorrect')

6
resources/recipes/nytimes_sub.recipe
View File

@ -282,9 +282,9 @@ class NYTimes(BasicNewsRecipe):
br = BasicNewsRecipe.get_browser()
if self.username is not None and self.password is not None:
br.open('http://www.nytimes.com/auth/login')
br.select_form(name='login')
br['USERID'] = self.username
br['PASSWORD'] = self.password
br.form = br.forms().next()
br['userid'] = self.username
br['password'] = self.password
raw = br.submit().read()
if 'Please try again' in raw:
raise Exception('Your username and password are incorrect')

35
resources/recipes/nzz_ger.recipe
View File

@ -1,6 +1,6 @@
__license__ = 'GPL v3'
__copyright__ = '2009, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2009-2010, Darko Miletic <darko.miletic at gmail.com>'
'''
www.nzz.ch
@ -20,6 +20,19 @@ class Nzz(BasicNewsRecipe):
encoding = 'utf-8'
use_embedded_content = False
language = 'de'
extra_css = """
body{font-family: Georgia,"Times New Roman",Times,serif }
.artikel h3,.artikel h4,.bildLegende,.question,.autor{font-family: Arial,Verdana,Helvetica,sans-serif}
.bildLegende{font-size: small}
.autor{font-size: 0.9375em; color: #666666}
.quote{font-size: large !important;
font-style: italic;
font-weight: normal !important;
border-bottom: 1px dotted #BFBFBF;
border-top: 1px dotted #BFBFBF;
line-height: 1.25em}
.quelle{color: #666666; font-style: italic; white-space: nowrap}
"""
conversion_options = {
'comments' : description
@ -28,12 +41,14 @@ class Nzz(BasicNewsRecipe):
,'publisher' : publisher
}
keep_only_tags = [dict(name='div', attrs={'class':'article'})]
keep_only_tags = [dict(name='div', attrs={'class':'zone'})]
remove_tags_before = dict(name='p', attrs={'class':'dachzeile'})
remove_tags_after=dict(name='p', attrs={'class':'fussnote'})
remove_attributes=['width','height','lang']
remove_tags = [
dict(name=['object','link','base'])
,dict(name='div',attrs={'class':['more','teaser','advXertXoriXals','legal']})
,dict(name='div',attrs={'id':['popup-src','readercomments','google-ad','advXertXoriXals']})
dict(name=['object','link','base','meta','iframe'])
,dict(attrs={'id':'content_rectangle_1'})
,dict(attrs={'class':['weiterfuehrendeLinks','fussnote','video']})
]
feeds = [
@ -50,7 +65,7 @@ class Nzz(BasicNewsRecipe):
,(u'Reisen' , u'http://www.nzz.ch/magazin/reisen?rss=true')
]
def print_version(self, url):
return url + '?printview=true'
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
return self.adeify_images(soup)

19
resources/recipes/poughkeepsie_journal.recipe Normal file
View File

@ -0,0 +1,19 @@
from calibre.web.feeds.news import BasicNewsRecipe
class AdvancedUserRecipe1291143841(BasicNewsRecipe):
title = u'Poughkeepsipe Journal'
language = 'en'
__author__ = 'weebl'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
timefmt = ' [%a, %d %b, %Y]'
feeds = [(u'Local News', u'http://poughkeepsiejournal.com/apps/pbcs.dll/oversikt?Category=RSS01&mime=xml'),
(u'Local Business', u'http://poughkeepsiejournal.com/apps/pbcs.dll/oversikt?Category=RSS02&mime=xml'),
(u'Local Sports', u'http://poughkeepsiejournal.com/apps/pbcs.dll/oversikt?Category=RSS03&mime=xml'),
(u'Life', u'http://poughkeepsiejournal.com/apps/pbcs.dll/oversikt?Category=RSS04&mime=xml')]
remove_tags = [dict(name='img', attrs={'src':'/graphics/mastlogo.gif'})]
def print_version(self, url):
return url.replace('http://www.poughkeepsiejournal.com', 'http://www.poughkeepsiejournal.com/print')

70
resources/recipes/st_louis_post_dispatch.recipe Normal file
View File

@ -0,0 +1,70 @@
from calibre.web.feeds.news import BasicNewsRecipe
class AdvancedUserRecipe1282093204(BasicNewsRecipe):
title = u'St Louis Post-Dispatch'
__author__ = 'cisaak'
language = 'en'
oldest_article = 1
max_articles_per_feed = 15
masthead_url = 'http://farm5.static.flickr.com/4118/4929686950_0e22e2c88a.jpg'
feeds = [
(u'News-Bill McClellan', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2fcolumns%2Fbill-mclellan&f=rss&t=article'),
(u'News-Columns', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2Fcolumns*&l=50&f=rss&t=article'),
(u'News-Crime & Courtshttp://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2Fcrime-and-courts&l=50&f=rss&t=article'),
(u'News-Deb Peterson', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2fcolumns%2Fdeb-peterson&f=rss&t=article'),
(u'News-Education', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2feducation&f=rss&t=article'),
(u'News-Government & Politics', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2fgovt-and-politics&f=rss&t=article'),
(u'News-Local', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal&f=rss&t=article'),
(u'News-Metro', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2fmetro&f=rss&t=article'),
(u'News-Metro East', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Flocal%2fillinois&f=rss&t=article'),
(u'News-Missouri Out State', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Fstate-and-regional%2FMissouri&l=50&f=rss&t=article'),
(u'Opinion-Colleen Carroll Campbell', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Fopinion%2fcolumns%2Fcolleen-carroll-campbell&f=rss&t=article'),
(u'Opinion-Editorial', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Fopinion%2feditorial&f=rss&t=article'),
(u'Opinion-Kevin Horrigan', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Fopinion%2fcolumns%2Fkevin-horrigan&f=rss&t=article'),
(u'Opinion-Mailbag', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=news%2Fopinion%2fmailbag&f=rss&t=article'),
(u'Business Columns-Savvy Consumer', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Fsavvy-consumer&l=100&f=rss&t=article'),
(u'Business Columns-Lager Heads', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Flager-heads&l=100&f=rss&t=article'),
(u'Business Columns-Job Watch', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Fjob-watch&l=100&f=rss&t=article'),
(u'Business Columns-Steve Geigerich', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Fsteve-giegerich&l=100&f=rss&t=article'),
(u'Business Columns-David Nicklaus', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Fdavid-nicklaus&l=100&f=rss&t=article'),
(u'Business Columns-Jim Gallagher', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Fjim-gallagher&l=100&f=rss&t=article'),
(u'Business Columns-Building Blocks', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fcolumns%2Fbuilding-blocks&l=100&f=rss&t=article'),
(u'Business', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business*l&l=100&f=rss&t=article'),
(u'Business-Technology', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Ftechnology&l=50&f=rss&t=article'),
(u'Business-National', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=business%2Fnational-and-international&l=50&f=rss&t=article'),
(u'Travel', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=travel*&l=100&f=rss&t=article'),
(u'Sports', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports*&f=rss&t=article'),
(u'Sports-Baseball', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fbaseball%2Fprofessional&l=100&f=rss&t=article'),
(u'Sports-Bernie Miklasz', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fcolumns%2Fbernie-miklasz&l=50&f=rss&t=article'),
(u'Sports-Bryan Burwell', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fcolumns%2Fbryan-burwell&l=50&f=rss&t=article'),
(u'Sports-College', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fcollege*&l=100&f=rss&t=article'),
(u'Sports-Dan Caesar', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fcolumns%2Fdan-caesar&l=50&f=rss&t=article'),
(u'Sports-Football', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Ffootball%2Fprofessional&l=100&f=rss&t=article'),
(u'Sports-Hockey', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fhockey%2Fprofessional&l=100&f=rss&t=article'),
(u'Sports-Illini', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fcollege%2Fillini&l=100&f=rss&t=article'),
(u'Sports-Jeff Gordon', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fcolumns%2Fjeff-gordon&l=100&f=rss&t=article'),
(u'Life & Style', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles&l=100&f=rss&t=article'),
(u'Life & Style-Debra Bass', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles%2Ffashion-and-style%2Fdebra-bass&l=100&f=rss&t=article'),
(u'Life & Style-Food and Cooking', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles%2Ffood-and-cooking&l=100&f=rss&t=article'),
(u'Life & Style-Health/Medicine/Fitness', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles%2Fhealth-med-fit&l=100&f=rss&t=article'),
(u'Life & Style-Joe Holleman', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles%2Fcolumns%2Fjoe-holleman&l=100&f=rss&t=article'),
(u'Life & Style-Steals-and-Deals', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles%2Fcolumns%2Fsteals-and-deals&l=100&f=rss&t=article'),
(u'Life & Style-Tim Townsend', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=lifestyles%2Ffaith-and-values%2Ftim-townsend&l=100&f=rss&t=article'),
(u'Entertainment', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment&l=100&f=rss&t=article'),
(u'Entertainment-Arts & Theatre', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Farts-and-theatre&l=100&f=rss&t=article'),
(u'Entertainment-Books & Literature', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Fbooks-and-literature&l=100&f=rss&t=article'),
(u'Entertainment-Dining', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=sports%2Fhockey%2Fprofessional&l=100&f=rss&t=article'),
(u'Entertainment-Events Calendar', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Fevents-calendar&l=100&f=rss&t=article'),
(u'Entertainment-Gail Pennington', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Ftelevision%2Fgail-pennington&l=100&f=rss&t=article'),
(u'Entertainment-Hip Hops', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Fdining%2Fbars-and-clubs-other%2Fhip-hops&l=100&f=rss&t=article'),
(u'Entertainment-House-O-Fun', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Fhouse-o-fun&l=100&f=rss&t=article'),
(u'Entertainment-Kevin C. Johnson', u'http://www2.stltoday.com/search/?q=&d1=&d2=&s=start_time&sd=desc&c=entertainment%2Fmusic%2Fkevin-johnson&l=100&f=rss&t=article')
]
remove_empty_feeds = True
remove_tags = [dict(name='div', attrs={'id':'blox-logo'}),dict(name='a')]
keep_only_tags = [dict(name='h1'), dict(name='p', attrs={'class':'byline'}), dict(name="div", attrs={'id':'blox-story-text'})]
extra_css = 'p {text-align: left;}'

8
resources/recipes/the_h.recipe
View File

@ -14,7 +14,7 @@ class TheHeiseOnline(BasicNewsRecipe):
oldest_article = 3
description = 'In association with Heise Online'
publisher = 'Heise Media UK Ltd.'
category = 'news, technology, security'
category = 'news, technology, security, OSS, internet'
max_articles_per_feed = 100
language = 'en'
encoding = 'utf-8'
@ -27,6 +27,12 @@ class TheHeiseOnline(BasicNewsRecipe):
feeds = [
(u'The H News Feed', u'http://www.h-online.com/news/atom.xml')
]
cover_url = 'http://www.h-online.com/icons/logo_theH.gif'
remove_tags = [
dict(id="logo"),
dict(id="footer")
]
def print_version(self, url):
return url + '?view=print'

68
resources/recipes/toyokeizai.recipe Normal file
View File

@ -0,0 +1,68 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Hiroshi Miura <miurahr@linux.com>'
'''
www.toyokeizai.net
'''
from calibre.web.feeds.news import BasicNewsRecipe
import re
class Toyokeizai(BasicNewsRecipe):
title = u'ToyoKeizai News'
__author__ = 'Hiroshi Miura'
oldest_article = 1
max_articles_per_feed = 50
description = 'Japanese traditional economy and business magazine, only for advanced subscribers supported'
publisher = 'Toyokeizai Shinbun Sha'
category = 'economy, magazine, japan'
language = 'ja'
encoding = 'euc-jp'
index = 'http://member.toyokeizai.net/news/'
remove_javascript = True
no_stylesheets = True
masthead_title = u'TOYOKEIZAI'
needs_subscription = True
timefmt = '[%y/%m/%d]'
recursions = 5
match_regexps =[ r'page/\d+']
keep_only_tags = [
dict(name='div', attrs={'class':['news']}),
dict(name='div', attrs={'class':["news_cont"]}),
dict(name='div', attrs={'class':["news_con"]}),
# dict(name='div', attrs={'class':["norightsMessage"]})
]
remove_tags = [{'class':"mt35 mgz"},
{'class':"mt20 newzia"},
{'class':"mt20 fontS"},
{'class':"bk_btn_m"},
dict(id='newzia_connect_member')
]
def parse_index(self):
feeds = []
soup = self.index_to_soup(self.index)
topstories = soup.find('ul',attrs={'class':'list6'})
if topstories:
newsarticles = []
for itt in topstories.findAll('li'):
itema = itt.find('a',href=True)
itemd = itt.find('span')
newsarticles.append({
'title' :itema.string
,'date' :re.compile(r"\- ").sub("",itemd.string)
,'url' :'http://member.toyokeizai.net' + itema['href']
,'description':itema['title']
})
feeds.append(('news', newsarticles))
return feeds
def get_browser(self):
br = BasicNewsRecipe.get_browser()
if self.username is not None and self.password is not None:
br.open('http://member.toyokeizai.net/norights/form/')
br.select_form(nr=0)
br['kaiin_id'] = self.username
br['password'] = self.password
br.submit()
return br

72
resources/recipes/tpm_uk.recipe Normal file
View File

@ -0,0 +1,72 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
'''
www.philosophypress.co.uk
'''
from calibre.web.feeds.news import BasicNewsRecipe
class TPM_uk(BasicNewsRecipe):
title = "The Philosophers' Magazine"
__author__ = 'Darko Miletic'
description = 'Title says it all'
publisher = "The Philosophers' Magazine"
category = 'philosophy, news'
oldest_article = 25
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'utf8'
use_embedded_content = False
language = 'en_GB'
remove_empty_feeds = True
publication_type = 'magazine'
masthead_url = 'http://www.philosophypress.co.uk/wp-content/themes/masterplan/tma/images/bg/sitelogo.png'
extra_css = """
body{font-family: Helvetica,Arial,"Lucida Grande",Verdana,sans-serif }
img{margin-bottom: 0.4em; display:block}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
remove_tags = [
dict(name=['meta','link','base','iframe','embed','object','img'])
,dict(attrs={'id':['respond','sharethis_0']})
,dict(attrs={'class':'wp-caption-text'})
]
keep_only_tags=[
dict(attrs={'class':['post_cat','post_name','post_meta','post_text']})
,dict(attrs={'id':'comments'})
]
remove_attributes=['lang','width','height']
feeds = [
(u'Columns' , u'http://www.philosophypress.co.uk/?feed=rss2&cat=15' )
,(u'Essays' , u'http://www.philosophypress.co.uk/?feed=rss2&cat=19' )
,(u"21'st Century" , u'http://www.philosophypress.co.uk/?feed=rss2&cat=101')
,(u'Interviews' , u'http://www.philosophypress.co.uk/?feed=rss2&cat=9' )
,(u'News' , u'http://www.philosophypress.co.uk/?feed=rss2&cat=28' )
,(u'Profiles' , u'http://www.philosophypress.co.uk/?feed=rss2&cat=59' )
,(u'Reviews' , u'http://www.philosophypress.co.uk/?feed=rss2&cat=12' )
]
def get_cover_url(self):
soup = self.index_to_soup('http://www.philosophypress.co.uk/')
for image in soup.findAll('img',title=True):
if image['title'].startswith('Click to Subscribe'):
return image['src']
return None
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for alink in soup.findAll('a', rel=True):
if alink.string is not None:
tstr = alink.string
alink.replaceWith(tstr)
return soup

37
resources/recipes/tr.recipe Normal file
View File

@ -0,0 +1,37 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Anton Gillert <atx at binaryninja.de>'
'''
Fetch Technology Review.
'''
from time import strftime
from calibre.web.feeds.news import BasicNewsRecipe
class TechnologyReviewDe(BasicNewsRecipe):
title = 'Technology Review'
description = 'Technology news from Germany'
__author__ = 'Anton Gillert'
use_embedded_content = False
language = 'de'
timefmt = ' [%d %b %Y]'
max_articles_per_feed = 40
no_stylesheets = True
feeds = [ ('Technology Review', 'http://www.heise.de/tr/news-atom.xml') ]
def print_version(self, url):
return url + '?view=print'
remove_tags = [dict(id='navi_top'),
dict(id='navi_bottom'),
dict(name='div', attrs={'class':'navi_top_logo'}),
dict(name='img', attrs={'src':'/tr/icons/tr_logo2006.gif'}),
dict(name='p', attrs={'class':'size80'})]
remove_tags_after = [dict(name='p', attrs={'class':'size80'})]
def get_cover_url(self):
return 'http://www.heise-medien.de/presseinfo/bilder/tr/' + strftime("%y/tr%m%Y.jpg")

2
resources/recipes/yomiuri.recipe
View File

@ -21,7 +21,7 @@ class YOLNews(BasicNewsRecipe):
remove_javascript = True
masthead_title = u'YOMIURI ONLINE'
remove_tags_before = {'class':"article-def"}
keep_only_tags = [{'class':"article-def"}]
remove_tags = [{'class':"RelatedArticle"},
{'class':"sbtns"}
]

2
resources/recipes/yomiuri_world.recipe
View File

@ -21,7 +21,7 @@ class YOLNews(BasicNewsRecipe):
remove_javascript = True
masthead_title = u"YOMIURI ONLINE"
remove_tags_before = {'class':"article-def"}
keep_only_tags = [{'class':"article-def"}]
remove_tags = [{'class':"RelatedArticle"},
{'class':"sbtns"}
]

2
session.vim
View File

@ -1,5 +1,5 @@
" Project wide builtins
let g:pyflakes_builtins += ["dynamic_property", "__", "P", "I", "lopen"]
let g:pyflakes_builtins += ["dynamic_property", "__", "P", "I", "lopen", "icu_lower", "icu_upper", "icu_title"]
python << EOFPY
import os

4
setup/build_environment.py
View File

@ -91,11 +91,15 @@ podofo_inc = '/usr/include/podofo'
podofo_lib = '/usr/lib'
chmlib_inc_dirs = chmlib_lib_dirs = []
sqlite_inc_dirs = []
icu_inc_dirs = []
icu_lib_dirs = []
if iswindows:
prefix = r'C:\cygwin\home\kovid\sw'
sw_inc_dir = os.path.join(prefix, 'include')
sw_lib_dir = os.path.join(prefix, 'lib')
icu_inc_dirs = [sw_inc_dir]
icu_lib_dirs = [sw_lib_dir]
sqlite_inc_dirs = [sw_inc_dir]
fc_inc = os.path.join(sw_inc_dir, 'fontconfig')
fc_lib = sw_lib_dir

3
setup/check.py
View File

@ -63,7 +63,8 @@ class Check(Command):
description = 'Check for errors in the calibre source code'
BUILTINS = ['_', '__', 'dynamic_property', 'I', 'P', 'lopen']
BUILTINS = ['_', '__', 'dynamic_property', 'I', 'P', 'lopen', 'icu_lower',
'icu_upper', 'icu_title']
CACHE = '.check-cache.pickle'
def get_files(self, cache):

20
setup/extensions.py
View File

@ -18,7 +18,8 @@ from setup.build_environment import fc_inc, fc_lib, chmlib_inc_dirs, \
QMAKE, msvc, MT, win_inc, win_lib, png_inc_dirs, win_ddk, \
magick_inc_dirs, magick_lib_dirs, png_lib_dirs, png_libs, \
magick_error, magick_libs, ft_lib_dirs, ft_libs, jpg_libs, \
jpg_lib_dirs, chmlib_lib_dirs, sqlite_inc_dirs
jpg_lib_dirs, chmlib_lib_dirs, sqlite_inc_dirs, icu_inc_dirs, \
icu_lib_dirs
MT
isunix = islinux or isosx or isfreebsd
@ -56,8 +57,25 @@ pdfreflow_libs = []
if iswindows:
pdfreflow_libs = ['advapi32', 'User32', 'Gdi32', 'zlib']
icu_libs = ['icudata', 'icui18n', 'icuuc', 'icuio']
icu_cflags = []
if iswindows:
icu_libs = ['icudt', 'icuin', 'icuuc', 'icuio']
if isosx:
icu_libs = ['icucore']
icu_cflags = ['-DU_DISABLE_RENAMING'] # Needed to use system libicucore.dylib
extensions = [
Extension('icu',
['calibre/utils/icu.c'],
libraries=icu_libs,
lib_dirs=icu_lib_dirs,
inc_dirs=icu_inc_dirs,
cflags=icu_cflags
),
Extension('sqlite_custom',
['calibre/library/sqlite_custom.c'],
inc_dirs=sqlite_inc_dirs

7
setup/installer/linux/freeze2.py
View File

@ -14,7 +14,8 @@ from setup import Command, modules, basenames, functions, __version__, \
SITE_PACKAGES = ['IPython', 'PIL', 'dateutil', 'dns', 'PyQt4', 'mechanize',
'sip.so', 'BeautifulSoup.py', 'cssutils', 'encutils', 'lxml',
'sipconfig.py', 'xdg']
'sipconfig.py', 'xdg', 'dbus', '_dbus_bindings.so', 'dbus_bindings.py',
'_dbus_glib_bindings.so']
QTDIR = '/usr/lib/qt4'
QTDLLS = ('QtCore', 'QtGui', 'QtNetwork', 'QtSvg', 'QtXml', 'QtWebKit', 'QtDBus')
@ -49,6 +50,10 @@ binary_includes = [
'/lib/libreadline.so.6',
'/usr/lib/libchm.so.0',
'/usr/lib/liblcms2.so.2',
'/usr/lib/libicudata.so.46',
'/usr/lib/libicui18n.so.46',
'/usr/lib/libicuuc.so.46',
'/usr/lib/libicuio.so.46',
]
binary_includes += [os.path.join(QTDIR, 'lib%s.so.4'%x) for x in QTDLLS]

2
setup/installer/windows/freeze.py
View File

@ -199,7 +199,7 @@ class Win32Freeze(Command, WixMixIn):
for pat in ('*.dll',):
for f in glob.glob(os.path.join(bindir, pat)):
ok = True
for ex in ('expatw',):
for ex in ('expatw', 'testplug'):
if ex in f.lower():
ok = False
if not ok: continue

9
setup/installer/windows/notes.rst
View File

@ -77,6 +77,15 @@ Test it on the target system with
calibre-debug -c "import _imaging, _imagingmath, _imagingft, _imagingcms"
ICU
-------
Download the win32 msvc9 binary from http://www.icu-project.org/download/4.4.html
Note that 4.4 is the last version of ICU that can be compiled (is precompiled) with msvc9
Put the dlls into sw/bin and the unicode dir into sw/include and the contents of lib int sw/lib
Libunrar
----------

5
src/calibre/constants.py
View File

@ -2,7 +2,7 @@ __license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'
__appname__ = 'calibre'
__version__ = '0.7.31'
__version__ = '0.7.33'
__author__ = "Kovid Goyal <kovid@kovidgoyal.net>"
import re
@ -67,7 +67,8 @@ if plugins is None:
'pdfreflow',
'progress_indicator',
'chmlib',
'chm_extra'
'chm_extra',
'icu',
] + \
(['winutil'] if iswindows else []) + \
(['usbobserver'] if isosx else []):

2
src/calibre/customize/profiles.py
View File

@ -37,6 +37,8 @@ class Plugin(_Plugin):
self.fsizes.append((name, num, float(size)))
self.fnames = dict((name, sz) for name, _, sz in self.fsizes if name)
self.fnums = dict((num, sz) for _, num, sz in self.fsizes if num)
self.width_pts = self.width * 72./self.dpi
self.height_pts = self.height * 72./self.dpi
# Input profiles {{{
class InputProfile(Plugin):

4
src/calibre/devices/android/driver.py
View File

@ -21,7 +21,7 @@ class ANDROID(USBMS):
# HTC
0x0bb4 : { 0x0c02 : [0x100, 0x0227, 0x0226], 0x0c01 : [0x100, 0x0227], 0x0ff9
: [0x0100, 0x0227, 0x0226], 0x0c87: [0x0100, 0x0227, 0x0226],
0xc92 : [0x100]},
0xc92 : [0x100], 0xc97: [0x226]},
# Eken
0x040d : { 0x8510 : [0x0001] },
@ -63,7 +63,7 @@ class ANDROID(USBMS):
WINDOWS_MAIN_MEM = ['ANDROID_PHONE', 'A855', 'A853', 'INC.NEXUS_ONE',
'__UMS_COMPOSITE', '_MB200', 'MASS_STORAGE', '_-_CARD', 'SGH-I897',
'GT-I9000', 'FILE-STOR_GADGET', 'SGH-T959', 'SAMSUNG_ANDROID',
'SCH-I500_CARD']
'SCH-I500_CARD', 'SPH-D700_CARD', 'MB810']
WINDOWS_CARD_A_MEM = ['ANDROID_PHONE', 'GT-I9000_CARD', 'SGH-I897',
'FILE-STOR_GADGET', 'SGH-T959', 'SAMSUNG_ANDROID']

8
src/calibre/devices/apple/driver.py
View File

@ -2637,7 +2637,7 @@ class ITUNES(DriverBase):
lb_added.composer.set(metadata_x.uuid)
lb_added.description.set("%s %s" % (self.description_prefix,strftime('%Y-%m-%d %H:%M:%S')))
lb_added.enabled.set(True)
lb_added.sort_artist.set(metadata_x.author_sort.title())
lb_added.sort_artist.set(icu_title(metadata_x.author_sort))
lb_added.sort_name.set(metadata.title_sort)
@ -2648,7 +2648,7 @@ class ITUNES(DriverBase):
db_added.composer.set(metadata_x.uuid)
db_added.description.set("%s %s" % (self.description_prefix,strftime('%Y-%m-%d %H:%M:%S')))
db_added.enabled.set(True)
db_added.sort_artist.set(metadata_x.author_sort.title())
db_added.sort_artist.set(icu_title(metadata_x.author_sort))
db_added.sort_name.set(metadata.title_sort)
if metadata_x.comments:
@ -2729,7 +2729,7 @@ class ITUNES(DriverBase):
lb_added.Composer = metadata_x.uuid
lb_added.Description = ("%s %s" % (self.description_prefix,strftime('%Y-%m-%d %H:%M:%S')))
lb_added.Enabled = True
lb_added.SortArtist = metadata_x.author_sort.title()
lb_added.SortArtist = icu_title(metadata_x.author_sort)
lb_added.SortName = metadata.title_sort
if db_added:
@ -2739,7 +2739,7 @@ class ITUNES(DriverBase):
db_added.Composer = metadata_x.uuid
db_added.Description = ("%s %s" % (self.description_prefix,strftime('%Y-%m-%d %H:%M:%S')))
db_added.Enabled = True
db_added.SortArtist = metadata_x.author_sort.title()
db_added.SortArtist = icu_title(metadata_x.author_sort)
db_added.SortName = metadata.title_sort
if metadata_x.comments:

1
src/calibre/devices/cybook/driver.py
View File

@ -65,6 +65,7 @@ class ORIZON(CYBOOK):
BCD = [0x319]
VENDOR_NAME = ['BOOKEEN', 'LINUX']
WINDOWS_MAIN_MEM = re.compile(r'(CYBOOK_ORIZON__-FD)|(FILE-STOR_GADGET)')
WINDOWS_CARD_A_MEM = re.compile('(CYBOOK_ORIZON__-SD)|(FILE-STOR_GADGET)')

19
src/calibre/devices/usbms/books.py
View File

@ -11,8 +11,9 @@ from calibre.ebooks.metadata.book.base import Metadata
from calibre.devices.mime import mime_type_ext
from calibre.devices.interface import BookList as _BookList
from calibre.constants import preferred_encoding
from calibre import isbytestring
from calibre import isbytestring, force_unicode
from calibre.utils.config import prefs, tweaks
from calibre.utils.icu import strcmp
class Book(Metadata):
def __init__(self, prefix, lpath, size=None, other=None):
@ -215,14 +216,17 @@ class CollectionsBookList(BookList):
elif is_series:
if doing_dc:
collections[cat_name][lpath] = \
(book, book.get('series_index', sys.maxint), '')
(book, book.get('series_index', sys.maxint),
book.get('title_sort', 'zzzz'))
else:
collections[cat_name][lpath] = \
(book, book.get(attr+'_index', sys.maxint), '')
(book, book.get(attr+'_index', sys.maxint),
book.get('title_sort', 'zzzz'))
else:
if lpath not in collections[cat_name]:
collections[cat_name][lpath] = \
(book, book.get('title_sort', 'zzzz'), '')
(book, book.get('title_sort', 'zzzz'),
book.get('title_sort', 'zzzz'))
# Sort collections
result = {}
@ -230,14 +234,19 @@ class CollectionsBookList(BookList):
x = xx[1]
y = yy[1]
if x is None and y is None:
# No sort_key needed here, because defaults are ascii
return cmp(xx[2], yy[2])
if x is None:
return 1
if y is None:
return -1
c = cmp(x, y)
if isinstance(x, basestring) and isinstance(y, basestring):
c = strcmp(force_unicode(x), force_unicode(y))
else:
c = cmp(x, y)
if c != 0:
return c
# same as above -- no sort_key needed here
return cmp(xx[2], yy[2])
for category, lpaths in collections.items():

3
src/calibre/ebooks/epub/output.py
View File

@ -142,6 +142,9 @@ class EPUBOutput(OutputFormatPlugin):
def convert(self, oeb, output_path, input_plugin, opts, log):
self.log, self.opts, self.oeb = log, opts, oeb
#from calibre.ebooks.oeb.transforms.filenames import UniqueFilenames
#UniqueFilenames()(oeb, opts)
self.workaround_ade_quirks()
self.workaround_webkit_quirks()
self.upshift_markup()

7
src/calibre/ebooks/epub/periodical.py
View File

@ -6,9 +6,11 @@ __copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
from uuid import uuid4
import time
from calibre.constants import __appname__, __version__
from calibre import strftime, prepare_string_for_xml as xml
from calibre.utils.date import parse_date
SONY_METADATA = u'''\
<?xml version="1.0" encoding="utf-8"?>
@ -87,7 +89,8 @@ def sony_metadata(oeb):
pass
try:
date = unicode(m.date[0]).split('T')[0]
date = parse_date(unicode(m.date[0]),
as_utc=False).strftime('%Y-%m-%d')
except:
date = strftime('%Y-%m-%d')
try:
@ -101,7 +104,7 @@ def sony_metadata(oeb):
publisher=xml(publisher), issue_date=xml(date),
language=xml(language))
updated = strftime('%Y-%m-%dT%H:%M:%SZ')
updated = strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
def cal_id(x):
for k, v in x.attrib.items():

443
src/calibre/ebooks/fb2/fb2ml.py
View File

@ -8,15 +8,11 @@ __docformat__ = 'restructuredtext en'
Transform OEB content into FB2 markup
'''
import cStringIO
from base64 import b64encode
from datetime import datetime
from mimetypes import types_map
import re
try:
from PIL import Image
Image
except ImportError:
import Image
import uuid
from lxml import etree
@ -24,41 +20,13 @@ from calibre import prepare_string_for_xml
from calibre.constants import __appname__, __version__
from calibre.ebooks.oeb.base import XHTML, XHTML_NS, barename, namespace
from calibre.ebooks.oeb.stylizer import Stylizer
from calibre.ebooks.oeb.base import OEB_RASTER_IMAGES
TAG_MAP = {
'b' : 'strong',
'i' : 'emphasis',
'p' : 'p',
'li' : 'p',
'div': 'p',
'br' : 'p',
}
TAG_SPACE = []
TAG_IMAGES = [
'img',
]
TAG_LINKS = [
'a',
]
BLOCK = [
'p',
]
STYLES = [
('font-weight', {'bold' : 'strong', 'bolder' : 'strong'}),
('font-style', {'italic' : 'emphasis'}),
]
from calibre.ebooks.oeb.base import OEB_RASTER_IMAGES, OPF
from calibre.utils.magick import Image
class FB2MLizer(object):
'''
Todo: * Ensure all style tags are inside of the p tags.
* Include more FB2 specific tags in the conversion.
* Handle reopening of a tag properly.
Todo: * Include more FB2 specific tags in the conversion.
* Handle a tags.
* Figure out some way to turn oeb_book.toc items into <section><title>
<p> to allow for readers to generate toc from the document.
'''
@ -66,29 +34,36 @@ class FB2MLizer(object):
def __init__(self, log):
self.log = log
self.image_hrefs = {}
self.link_hrefs = {}
self.reset_state()
def reset_state(self):
# Used to ensure text and tags are always within <p> and </p>
self.in_p = False
# Mapping of image names. OEB allows for images to have the same name but be stored
# in different directories. FB2 images are all in a flat layout so we rename all images
# into a sequential numbering system to ensure there are no collisions between image names.
self.image_hrefs = {}
def extract_content(self, oeb_book, opts):
self.log.info('Converting XHTML to FB2 markup...')
self.oeb_book = oeb_book
self.opts = opts
return self.fb2mlize_spine()
def fb2mlize_spine(self):
self.image_hrefs = {}
self.link_hrefs = {}
self.reset_state()
output = [self.fb2_header()]
output.append(self.get_cover_page())
output.append(u'ghji87yhjko0Caliblre-toc-placeholder-for-insertion-later8ujko0987yjk')
output.append(self.get_text())
output.append(self.fb2_body_footer())
output.append(self.fb2mlize_images())
output.append(self.fb2_footer())
output = ''.join(output).replace(u'ghji87yhjko0Caliblre-toc-placeholder-for-insertion-later8ujko0987yjk', self.get_toc())
output = self.clean_text(output)
if self.opts.sectionize_chapters:
output = self.sectionize_chapters(output)
return u'<?xml version="1.0" encoding="UTF-8"?>\n%s' % etree.tostring(etree.fromstring(output), encoding=unicode, pretty_print=True)
output = self.clean_text(u''.join(output))
if self.opts.pretty_print:
return u'<?xml version="1.0" encoding="UTF-8"?>\n%s' % etree.tostring(etree.fromstring(output), encoding=unicode, pretty_print=True)
else:
return u'<?xml version="1.0" encoding="UTF-8"?>' + output
def clean_text(self, text):
text = re.sub(r'(?miu)<section>\s*</section>', '', text)
@ -101,113 +76,94 @@ class FB2MLizer(object):
return text
def fb2_header(self):
author_first = u''
author_middle = u''
author_last = u''
metadata = {}
metadata['author_first'] = u''
metadata['author_middle'] = u''
metadata['author_last'] = u''
metadata['title'] = self.oeb_book.metadata.title[0].value
metadata['appname'] = __appname__
metadata['version'] = __version__
metadata['date'] = '%i.%i.%i' % (datetime.now().day, datetime.now().month, datetime.now().year)
metadata['lang'] = u''.join(self.oeb_book.metadata.lang) if self.oeb_book.metadata.lang else 'en'
metadata['id'] = None
author_parts = self.oeb_book.metadata.creator[0].value.split(' ')
if len(author_parts) == 1:
author_last = author_parts[0]
metadata['author_last'] = author_parts[0]
elif len(author_parts) == 2:
author_first = author_parts[0]
author_last = author_parts[1]
metadata['author_first'] = author_parts[0]
metadata['author_last'] = author_parts[1]
else:
author_first = author_parts[0]
author_middle = ' '.join(author_parts[1:-2])
author_last = author_parts[-1]
metadata['author_first'] = author_parts[0]
metadata['author_middle'] = ' '.join(author_parts[1:-2])
metadata['author_last'] = author_parts[-1]
return u'<FictionBook xmlns:xlink="http://www.w3.org/1999/xlink" ' \
'xmlns="http://www.gribuser.ru/xml/fictionbook/2.0">\n' \
'<description>\n<title-info>\n ' \
'<author>\n<first-name>%s</first-name>\n<middle-name>%s' \
'</middle-name>\n<last-name>%s</last-name>\n</author>\n' \
'<book-title>%s</book-title> ' \
'</title-info><document-info> ' \
'<program-used>%s - %s</program-used></document-info>\n' \
'</description>\n<body>\n<section>' % tuple(map(prepare_string_for_xml,
(author_first, author_middle,
author_last, self.oeb_book.metadata.title[0].value,
__appname__, __version__)))
identifiers = self.oeb_book.metadata['identifier']
for x in identifiers:
if x.get(OPF('scheme'), None).lower() == 'uuid' or unicode(x).startswith('urn:uuid:'):
metadata['id'] = unicode(x).split(':')[-1]
break
if metadata['id'] is None:
self.log.warn('No UUID identifier found')
metadata['id'] = str(uuid.uuid4())
def get_cover_page(self):
output = u''
if 'cover' in self.oeb_book.guide:
output += '<image xlink:href="#cover.jpg" />'
self.image_hrefs[self.oeb_book.guide['cover'].href] = 'cover.jpg'
if 'titlepage' in self.oeb_book.guide:
self.log.debug('Generating cover page...')
href = self.oeb_book.guide['titlepage'].href
item = self.oeb_book.manifest.hrefs[href]
if item.spine_position is None:
stylizer = Stylizer(item.data, item.href, self.oeb_book,
self.opts, self.opts.output_profile)
output += ''.join(self.dump_text(item.data.find(XHTML('body')), stylizer, item))
return output
for key, value in metadata.items():
metadata[key] = prepare_string_for_xml(value)
def get_toc(self):
toc = []
if self.opts.inline_toc:
self.log.debug('Generating table of contents...')
toc.append(u'<p>%s</p>' % _('Table of Contents:'))
for item in self.oeb_book.toc:
if item.href in self.link_hrefs.keys():
toc.append('<p><a xlink:href="#%s">%s</a></p>\n' % (self.link_hrefs[item.href], item.title))
else:
self.oeb.warn('Ignoring toc item: %s not found in document.' % item)
return ''.join(toc)
def sectionize_chapters(self, text):
def remove_p(t):
t = t.replace('<p>', '')
t = t.replace('</p>', '')
return t
text = re.sub(r'(?imsu)(<p>)\s*(?P<anchor><a\s+id="calibre_link-\d+"\s*/>)\s*(</p>)\s*(<p>)\s*(?P<strong><strong>.+?</strong>)\s*(</p>)', lambda mo: '</section><section>%s<title><p>%s</p></title>' % (mo.group('anchor'), remove_p(mo.group('strong'))), text)
text = re.sub(r'(?imsu)(<p>)\s*(?P<anchor><a\s+id="calibre_link-\d+"\s*/>)\s*(</p>)\s*(?P<strong><strong>.+?</strong>)', lambda mo: '</section><section>%s<title><p>%s</p></title>' % (mo.group('anchor'), remove_p(mo.group('strong'))), text)
text = re.sub(r'(?imsu)(?P<anchor><a\s+id="calibre_link-\d+"\s*/>)\s*(<p>)\s*(?P<strong><strong>.+?</strong>)\s*(</p>)', lambda mo: '</section><section>%s<title><p>%s</p></title>' % (mo.group('anchor'), remove_p(mo.group('strong'))), text)
text = re.sub(r'(?imsu)(<p>)\s*(?P<anchor><a\s+id="calibre_link-\d+"\s*/>)\s*(?P<strong><strong>.+?</strong>)\s*(</p>)', lambda mo: '</section><section>%s<title><p>%s</p></title>' % (mo.group('anchor'), remove_p(mo.group('strong'))), text)
text = re.sub(r'(?imsu)(?P<anchor><a\s+id="calibre_link-\d+"\s*/>)\s*(?P<strong><strong>.+?</strong>)', lambda mo: '</section><section>%s<title><p>%s</p></title>' % (mo.group('anchor'), remove_p(mo.group('strong'))), text)
return text
def get_text(self):
text = []
for i, item in enumerate(self.oeb_book.spine):
if self.opts.sectionize_chapters_using_file_structure and i is not 0:
text.append('<section>')
self.log.debug('Converting %s to FictionBook2 XML' % item.href)
stylizer = Stylizer(item.data, item.href, self.oeb_book, self.opts, self.opts.output_profile)
text.append(self.add_page_anchor(item))
text += self.dump_text(item.data.find(XHTML('body')), stylizer, item)
if self.opts.sectionize_chapters_using_file_structure and i is not len(self.oeb_book.spine) - 1:
text.append('</section>')
return ''.join(text)
def fb2_body_footer(self):
return u'\n</section>\n</body>'
return u'<FictionBook xmlns="http://www.gribuser.ru/xml/fictionbook/2.0" xmlns:xlink="http://www.w3.org/1999/xlink">' \
'<description>' \
'<title-info>' \
'<genre>antique</genre>' \
'<author>' \
'<first-name>%(author_first)s</first-name>' \
'<middle-name>%(author_middle)s</middle-name>' \
'<last-name>%(author_last)s</last-name>' \
'</author>' \
'<book-title>%(title)s</book-title>' \
'<lang>%(lang)s</lang>' \
'</title-info>' \
'<document-info>' \
'<author>' \
'<first-name></first-name>' \
'<middle-name></middle-name>' \
'<last-name></last-name>' \
'</author>' \
'<program-used>%(appname)s %(version)s</program-used>' \
'<date>%(date)s</date>' \
'<id>%(id)s</id>' \
'<version>1.0</version>' \
'</document-info>' \
'</description>' % metadata
def fb2_footer(self):
return u'</FictionBook>'
def add_page_anchor(self, page):
return self.get_anchor(page, '')
def get_anchor(self, page, aid):
aid = prepare_string_for_xml(aid)
aid = '%s#%s' % (page.href, aid)
if aid not in self.link_hrefs.keys():
self.link_hrefs[aid] = 'calibre_link-%s' % len(self.link_hrefs.keys())
aid = self.link_hrefs[aid]
return '<a id="%s" />' % aid
def get_text(self):
text = ['<body>']
for item in self.oeb_book.spine:
self.log.debug('Converting %s to FictionBook2 XML' % item.href)
stylizer = Stylizer(item.data, item.href, self.oeb_book, self.opts, self.opts.output_profile)
text.append('<section>')
text += self.dump_text(item.data.find(XHTML('body')), stylizer, item)
text.append('</section>')
return ''.join(text) + '</body>'
def fb2mlize_images(self):
'''
This function uses the self.image_hrefs dictionary mapping. It is populated by the dump_text function.
'''
images = []
for item in self.oeb_book.manifest:
# Don't write the image if it's not referenced in the document's text.
if item.href not in self.image_hrefs:
continue
if item.media_type in OEB_RASTER_IMAGES:
try:
im = Image.open(cStringIO.StringIO(item.data)).convert('RGB')
data = cStringIO.StringIO()
im.save(data, 'JPEG')
data = data.getvalue()
if not item.media_type == types_map['.jpeg'] or not item.media_type == types_map['.jpg']:
im = Image()
im.load(item.data)
im.set_compression_quality(70)
data = im.export('jpg')
raw_data = b64encode(data)
# Don't put the encoded image on a single line.
data = ''
@ -218,114 +174,167 @@ class FB2MLizer(object):
col = 1
col += 1
data += char
images.append('<binary id="%s" content-type="%s">%s\n</binary>' % (self.image_hrefs.get(item.href, '0000.JPEG'), item.media_type, data))
images.append('<binary id="%s" content-type="image/jpeg">%s\n</binary>' % (self.image_hrefs[item.href], data))
except Exception as e:
self.log.error('Error: Could not include file %s becuase ' \
self.log.error('Error: Could not include file %s because ' \
'%s.' % (item.href, e))
return ''.join(images)
def dump_text(self, elem, stylizer, page, tag_stack=[]):
if not isinstance(elem.tag, basestring) \
or namespace(elem.tag) != XHTML_NS:
def ensure_p(self):
if self.in_p:
return [], []
else:
self.in_p = True
return ['<p>'], ['p']
def close_open_p(self, tags):
text = ['']
added_p = False
if self.in_p:
# Close all up to p. Close p. Reopen all closed tags including p.
closed_tags = []
tags.reverse()
for t in tags:
text.append('</%s>' % t)
closed_tags.append(t)
if t == 'p':
break
closed_tags.reverse()
for t in closed_tags:
text.append('<%s>' % t)
else:
text.append('<p>')
added_p = True
self.in_p = True
return text, added_p
def handle_simple_tag(self, tag, tags):
s_out = []
s_tags = []
if tag not in tags:
p_out, p_tags = self.ensure_p()
s_out += p_out
s_tags += p_tags
s_out.append('<%s>' % tag)
s_tags.append(tag)
return s_out, s_tags
def dump_text(self, elem_tree, stylizer, page, tag_stack=[]):
'''
This function is intended to be used in a recursive manner. dump_text will
run though all elements in the elem_tree and call itself on each element.
self.image_hrefs will be populated by calling this function.
@param elem_tree: etree representation of XHTML content to be transformed.
@param stylizer: Used to track the style of elements within the tree.
@param page: OEB page used to determine absolute urls.
@param tag_stack: List of open FB2 tags to take into account.
@return: List of string representing the XHTML converted to FB2 markup.
'''
# Ensure what we are converting is not a string and that the fist tag is part of the XHTML namespace.
if not isinstance(elem_tree.tag, basestring) or namespace(elem_tree.tag) != XHTML_NS:
return []
style = stylizer.style(elem)
if style['display'] in ('none', 'oeb-page-head', 'oeb-page-foot') \
or style['visibility'] == 'hidden':
style = stylizer.style(elem_tree)
if style['display'] in ('none', 'oeb-page-head', 'oeb-page-foot') or style['visibility'] == 'hidden':
return []
fb2_text = []
# FB2 generated output.
fb2_out = []
# FB2 tags in the order they are opened. This will be used to close the tags.
tags = []
# First tag in tree
tag = barename(elem_tree.tag)
tag = barename(elem.tag)
if tag in TAG_IMAGES:
if elem.attrib.get('src', None):
if page.abshref(elem.attrib['src']) not in self.image_hrefs.keys():
self.image_hrefs[page.abshref(elem.attrib['src'])] = '%s.jpg' % len(self.image_hrefs.keys())
fb2_text.append('<image xlink:href="#%s" />' % self.image_hrefs[page.abshref(elem.attrib['src'])])
if tag in TAG_LINKS:
href = elem.get('href')
if href:
href = prepare_string_for_xml(page.abshref(href))
href = href.replace('"', '&quot;')
if '://' in href:
fb2_text.append('<a xlink:href="%s">' % href)
else:
if href.startswith('#'):
href = href[1:]
if href not in self.link_hrefs.keys():
self.link_hrefs[href] = 'calibre_link-%s' % len(self.link_hrefs.keys())
href = self.link_hrefs[href]
fb2_text.append('<a xlink:href="#%s">' % href)
tags.append('a')
# Anchor ids
id_name = elem.get('id')
if id_name:
fb2_text.append(self.get_anchor(page, id_name))
# Process the XHTML tag if it needs to be converted to an FB2 tag.
if tag == 'h1' and self.opts.h1_to_title or tag == 'h2' and self.opts.h2_to_title or tag == 'h3' and self.opts.h3_to_title:
fb2_text.append('<title>')
fb2_out.append('<title>')
tags.append('title')
fb2_tag = TAG_MAP.get(tag, None)
if fb2_tag == 'p':
if 'p' in tag_stack+tags:
# Close all up to p. Close p. Reopen all closed tags including p.
all_tags = tag_stack+tags
if tag == 'img':
if elem_tree.attrib.get('src', None):
# Only write the image tag if it is in the manifest.
if page.abshref(elem_tree.attrib['src']) in self.oeb_book.manifest.hrefs.keys():
if page.abshref(elem_tree.attrib['src']) not in self.image_hrefs.keys():
self.image_hrefs[page.abshref(elem_tree.attrib['src'])] = '_%s.jpg' % len(self.image_hrefs.keys())
p_txt, p_tag = self.ensure_p()
fb2_out += p_txt
tags += p_tag
fb2_out.append('<image xlink:href="#%s" />' % self.image_hrefs[page.abshref(elem_tree.attrib['src'])])
elif tag == 'br':
if self.in_p:
closed_tags = []
all_tags.reverse()
for t in all_tags:
fb2_text.append('</%s>' % t)
open_tags = tag_stack+tags
open_tags.reverse()
for t in open_tags:
fb2_out.append('</%s>' % t)
closed_tags.append(t)
if t == 'p':
break
fb2_out.append('<empty-line />')
closed_tags.reverse()
for t in closed_tags:
fb2_text.append('<%s>' % t)
fb2_out.append('<%s>' % t)
else:
fb2_text.append('<p>')
fb2_out.append('<empty-line />')
elif tag in ('div', 'li', 'p'):
p_text, added_p = self.close_open_p(tag_stack+tags)
fb2_out += p_text
if added_p:
tags.append('p')
elif fb2_tag and fb2_tag not in tag_stack+tags:
fb2_text.append('<%s>' % fb2_tag)
tags.append(fb2_tag)
elif tag == 'b':
s_out, s_tags = self.handle_simple_tag('strong', tag_stack+tags)
fb2_out += s_out
tags += s_tags
elif tag == 'i':
s_out, s_tags = self.handle_simple_tag('emphasis', tag_stack+tags)
fb2_out += s_out
tags += s_tags
# Processes style information
for s in STYLES:
style_tag = s[1].get(style[s[0]], None)
if style_tag and style_tag not in tag_stack+tags:
fb2_text.append('<%s>' % style_tag)
tags.append(style_tag)
# Processes style information.
if style['font-style'] == 'italic':
s_out, s_tags = self.handle_simple_tag('emphasis', tag_stack+tags)
fb2_out += s_out
tags += s_tags
elif style['font-weight'] in ('bold', 'bolder'):
s_out, s_tags = self.handle_simple_tag('strong', tag_stack+tags)
fb2_out += s_out
tags += s_tags
if tag in TAG_SPACE:
if not fb2_text or fb2_text[-1] != ' ' or not fb2_text[-1].endswith(' '):
fb2_text.append(' ')
# Process element text.
if hasattr(elem_tree, 'text') and elem_tree.text:
if not self.in_p:
fb2_out.append('<p>')
fb2_out.append(prepare_string_for_xml(elem_tree.text))
if not self.in_p:
fb2_out.append('</p>')
if hasattr(elem, 'text') and elem.text:
if 'p' not in tag_stack+tags:
fb2_text.append('<p>%s</p>' % prepare_string_for_xml(elem.text))
else:
fb2_text.append(prepare_string_for_xml(elem.text))
for item in elem:
fb2_text += self.dump_text(item, stylizer, page, tag_stack+tags)
# Process sub-elements.
for item in elem_tree:
fb2_out += self.dump_text(item, stylizer, page, tag_stack+tags)
# Close open FB2 tags.
tags.reverse()
fb2_text += self.close_tags(tags)
fb2_out += self.close_tags(tags)
if hasattr(elem, 'tail') and elem.tail:
if 'p' not in tag_stack:
fb2_text.append('<p>%s</p>' % prepare_string_for_xml(elem.tail))
else:
fb2_text.append(prepare_string_for_xml(elem.tail))
# Process element text that comes after the close of the XHTML tag but before the next XHTML tag.
if hasattr(elem_tree, 'tail') and elem_tree.tail:
if not self.in_p:
fb2_out.append('<p>')
fb2_out.append(prepare_string_for_xml(elem_tree.tail))
if not self.in_p:
fb2_out.append('</p>')
return fb2_text
return fb2_out
def close_tags(self, tags):
text = []
for tag in tags:
text.append('</%s>' % tag)
if tag == 'p':
self.in_p = False
return text

22
src/calibre/ebooks/fb2/output.py
View File

@ -16,20 +16,6 @@ class FB2Output(OutputFormatPlugin):
file_type = 'fb2'
options = set([
OptionRecommendation(name='inline_toc',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Add Table of Contents to beginning of the book.')),
OptionRecommendation(name='sectionize_chapters',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Try to turn chapters into individual sections. ' \
'WARNING: ' \
'This option is experimental. It can cause conversion ' \
'to fail. It can also produce unexpected output.')),
OptionRecommendation(name='sectionize_chapters_using_file_structure',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Try to turn chapters into individual sections using the ' \
'internal structure of the ebook. This works well for EPUB ' \
'books that have been internally split by chapter.')),
OptionRecommendation(name='h1_to_title',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Wrap all h1 tags with fb2 title elements.')),
@ -43,6 +29,14 @@ class FB2Output(OutputFormatPlugin):
def convert(self, oeb_book, output_path, input_plugin, opts, log):
from calibre.ebooks.oeb.transforms.jacket import linearize_jacket
from calibre.ebooks.oeb.transforms.rasterize import SVGRasterizer, Unavailable
try:
rasterizer = SVGRasterizer()
rasterizer(oeb_book, opts)
except Unavailable:
self.log.warn('SVG rasterizer unavailable, SVG will not be converted')
linearize_jacket(oeb_book)
fb2mlizer = FB2MLizer(log)

5
src/calibre/ebooks/markdown/mdx_toc.py
View File

@ -18,9 +18,10 @@ def extract_alphanumeric(in_str=None):
"""
# I'm sure this is really inefficient and
# could be done with a lambda/map()
#x.strip().title().replace(' ', "")
#x.strip(). title().replace(' ', "")
out_str=[]
for x in in_str.title():
for x in in_str:
x = icu_title(x)
if x.isalnum(): out_str.append(x)
return ''.join(out_str)

516
src/calibre/ebooks/metadata/amazonfr.py Normal file
View File

@ -0,0 +1,516 @@
from __future__ import with_statement
__license__ = 'GPL 3'
__copyright__ = '2010, sengian <sengian1@gmail.com>'
import sys, textwrap, re, traceback
from urllib import urlencode
from math import ceil
from lxml import html
from lxml.html import soupparser
from calibre.utils.date import parse_date, utcnow, replace_months
from calibre.utils.cleantext import clean_ascii_chars
from calibre import browser, preferred_encoding
from calibre.ebooks.chardet import xml_to_unicode
from calibre.ebooks.metadata import MetaInformation, check_isbn, \
authors_to_sort_string
from calibre.ebooks.metadata.fetch import MetadataSource
from calibre.utils.config import OptionParser
from calibre.library.comments import sanitize_comments_html
class AmazonFr(MetadataSource):
name = 'Amazon French'
description = _('Downloads metadata from amazon.fr')
supported_platforms = ['windows', 'osx', 'linux']
author = 'Sengian'
version = (1, 0, 0)
has_html_comments = True
def fetch(self):
try:
self.results = search(self.title, self.book_author, self.publisher,
self.isbn, max_results=10, verbose=self.verbose, lang='fr')
except Exception, e:
self.exception = e
self.tb = traceback.format_exc()
class AmazonEs(MetadataSource):
name = 'Amazon Spanish'
description = _('Downloads metadata from amazon.com in spanish')
supported_platforms = ['windows', 'osx', 'linux']
author = 'Sengian'
version = (1, 0, 0)
has_html_comments = True
def fetch(self):
try:
self.results = search(self.title, self.book_author, self.publisher,
self.isbn, max_results=10, verbose=self.verbose, lang='es')
except Exception, e:
self.exception = e
self.tb = traceback.format_exc()
class AmazonEn(MetadataSource):
name = 'Amazon English'
description = _('Downloads metadata from amazon.com in english')
supported_platforms = ['windows', 'osx', 'linux']
author = 'Sengian'
version = (1, 0, 0)
has_html_comments = True
def fetch(self):
try:
self.results = search(self.title, self.book_author, self.publisher,
self.isbn, max_results=10, verbose=self.verbose, lang='en')
except Exception, e:
self.exception = e
self.tb = traceback.format_exc()
class AmazonDe(MetadataSource):
name = 'Amazon German'
description = _('Downloads metadata from amazon.de')
supported_platforms = ['windows', 'osx', 'linux']
author = 'Sengian'
version = (1, 0, 0)
has_html_comments = True
def fetch(self):
try:
self.results = search(self.title, self.book_author, self.publisher,
self.isbn, max_results=10, verbose=self.verbose, lang='de')
except Exception, e:
self.exception = e
self.tb = traceback.format_exc()
class Amazon(MetadataSource):
name = 'Amazon'
description = _('Downloads metadata from amazon.com')
supported_platforms = ['windows', 'osx', 'linux']
author = 'Kovid Goyal & Sengian'
version = (1, 1, 0)
has_html_comments = True
def fetch(self):
# if not self.site_customization:
# return
try:
self.results = search(self.title, self.book_author, self.publisher,
self.isbn, max_results=10, verbose=self.verbose, lang='all')
except Exception, e:
self.exception = e
self.tb = traceback.format_exc()
# @property
# def string_customization_help(self):
# return _('You can select here the language for metadata search with amazon.com')
def report(verbose):
if verbose:
traceback.print_exc()
class Query(object):
BASE_URL_ALL = 'http://www.amazon.com'
BASE_URL_FR = 'http://www.amazon.fr'
BASE_URL_DE = 'http://www.amazon.de'
def __init__(self, title=None, author=None, publisher=None, isbn=None, keywords=None,
max_results=20, rlang='all'):
assert not(title is None and author is None and publisher is None \
and isbn is None and keywords is None)
assert (max_results < 21)
self.max_results = int(max_results)
self.renbres = re.compile(u'\s*(\d+)\s*')
q = { 'search-alias' : 'stripbooks' ,
'unfiltered' : '1',
'field-keywords' : '',
'field-author' : '',
'field-title' : '',
'field-isbn' : '',
'field-publisher' : ''
#get to amazon detailed search page to get all options
# 'node' : '',
# 'field-binding' : '',
#before, during, after
# 'field-dateop' : '',
#month as number
# 'field-datemod' : '',
# 'field-dateyear' : '',
#french only
# 'field-collection' : '',
#many options available
}
if rlang =='all':
q['sort'] = 'relevanceexprank'
self.urldata = self.BASE_URL_ALL
elif rlang =='es':
q['sort'] = 'relevanceexprank'
q['field-language'] = 'Spanish'
self.urldata = self.BASE_URL_ALL
elif rlang =='en':
q['sort'] = 'relevanceexprank'
q['field-language'] = 'English'
self.urldata = self.BASE_URL_ALL
elif rlang =='fr':
q['sort'] = 'relevancerank'
self.urldata = self.BASE_URL_FR
elif rlang =='de':
q['sort'] = 'relevancerank'
self.urldata = self.BASE_URL_DE
self.baseurl = self.urldata
if isbn is not None:
q['field-isbn'] = isbn.replace('-', '')
else:
if title is not None:
q['field-title'] = title
if author is not None:
q['field-author'] = author
if publisher is not None:
q['field-publisher'] = publisher
if keywords is not None:
q['field-keywords'] = keywords
if isinstance(q, unicode):
q = q.encode('utf-8')
self.urldata += '/gp/search/ref=sr_adv_b/?' + urlencode(q)
def __call__(self, browser, verbose, timeout = 5.):
if verbose:
print 'Query:', self.urldata
try:
raw = browser.open_novisit(self.urldata, timeout=timeout).read()
except Exception, e:
report(verbose)
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return
raise
if '<title>404 - ' in raw:
return
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
try:
feed = soupparser.fromstring(raw)
except:
try:
#remove ASCII invalid chars
return soupparser.fromstring(clean_ascii_chars(raw))
except:
return None, self.urldata
#nb of page
try:
nbresults = self.renbres.findall(feed.xpath("//*[@class='resultCount']")[0].text)
except:
return None, self.urldata
pages =[feed]
if len(nbresults) > 1:
nbpagetoquery = int(ceil(float(min(int(nbresults[2]), self.max_results))/ int(nbresults[1])))
for i in xrange(2, nbpagetoquery + 1):
try:
urldata = self.urldata + '&page=' + str(i)
raw = browser.open_novisit(urldata, timeout=timeout).read()
except Exception, e:
continue
if '<title>404 - ' in raw:
continue
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
try:
feed = soupparser.fromstring(raw)
except:
try:
#remove ASCII invalid chars
return soupparser.fromstring(clean_ascii_chars(raw))
except:
continue
pages.append(feed)
results = []
for x in pages:
results.extend([i.getparent().get('href') \
for i in x.xpath("//a/span[@class='srTitle']")])
return results[:self.max_results], self.baseurl
class ResultList(list):
def __init__(self, baseurl, lang = 'all'):
self.baseurl = baseurl
self.lang = lang
self.repub = re.compile(u'\((.*)\)')
self.rerat = re.compile(u'([0-9.]+)')
self.reattr = re.compile(r'<([a-zA-Z0-9]+)\s[^>]+>')
self.reoutp = re.compile(r'(?s)<em>--This text ref.*?</em>')
self.recom = re.compile(r'(?s)<!--.*?-->')
self.republi = re.compile(u'(Editeur|Publisher|Verlag)', re.I)
self.reisbn = re.compile(u'(ISBN-10|ISBN-10|ASIN)', re.I)
self.relang = re.compile(u'(Language|Langue|Sprache)', re.I)
self.reratelt = re.compile(u'(Average\s*Customer\s*Review|Moyenne\s*des\s*commentaires\s*client|Durchschnittliche\s*Kundenbewertung)', re.I)
self.reprod = re.compile(u'(Product\s*Details|D.tails\s*sur\s*le\s*produit|Produktinformation)', re.I)
def strip_tags_etree(self, etreeobj, invalid_tags):
for (itag, rmv) in invalid_tags.iteritems():
if rmv:
for elts in etreeobj.getiterator(itag):
elts.drop_tree()
else:
for elts in etreeobj.getiterator(itag):
elts.drop_tag()
def clean_entry(self, entry, invalid_tags = {'script': True},
invalid_id = (), invalid_class=()):
#invalid_tags: remove tag and keep content if False else remove
#remove tags
if invalid_tags:
self.strip_tags_etree(entry, invalid_tags)
#remove id
if invalid_id:
for eltid in invalid_id:
elt = entry.get_element_by_id(eltid)
if elt is not None:
elt.drop_tree()
#remove class
if invalid_class:
for eltclass in invalid_class:
elts = entry.find_class(eltclass)
if elts is not None:
for elt in elts:
elt.drop_tree()
def get_title(self, entry):
title = entry.get_element_by_id('btAsinTitle')
if title is not None:
title = title.text
return unicode(title.replace('\n', '').strip())
def get_authors(self, entry):
author = entry.get_element_by_id('btAsinTitle')
while author.getparent().tag != 'div':
author = author.getparent()
author = author.getparent()
authortext = []
for x in author.getiterator('a'):
authortext.append(unicode(x.text_content().strip()))
return authortext
def get_description(self, entry, verbose):
try:
description = entry.get_element_by_id("productDescription").find("div[@class='content']")
inv_class = ('seeAll', 'emptyClear')
inv_tags ={'img': True, 'a': False}
self.clean_entry(description, invalid_tags=inv_tags, invalid_class=inv_class)
description = html.tostring(description, method='html', encoding=unicode).strip()
# remove all attributes from tags
description = self.reattr.sub(r'<\1>', description)
# Remove the notice about text referring to out of print editions
description = self.reoutp.sub('', description)
# Remove comments
description = self.recom.sub('', description)
return unicode(sanitize_comments_html(description))
except:
report(verbose)
return None
def get_tags(self, entry, browser, verbose):
try:
tags = entry.get_element_by_id('tagContentHolder')
testptag = tags.find_class('see-all')
if testptag:
for x in testptag:
alink = x.xpath('descendant-or-self::a')
if alink:
if alink[0].get('class') == 'tgJsActive':
continue
link = self.baseurl + alink[0].get('href')
entry = self.get_individual_metadata(browser, link, verbose)
tags = entry.get_element_by_id('tagContentHolder')
break
tags = [a.text for a in tags.getiterator('a') if a.get('rel') == 'tag']
except:
report(verbose)
tags = []
return tags
def get_book_info(self, entry, mi, verbose):
try:
entry = entry.get_element_by_id('SalesRank').getparent()
except:
try:
for z in entry.getiterator('h2'):
if self.reprod.search(z.text_content()):
entry = z.getparent().find("div[@class='content']/ul")
break
except:
report(verbose)
return mi
elts = entry.findall('li')
#pub & date
elt = filter(lambda x: self.republi.search(x.find('b').text), elts)
if elt:
pub = elt[0].find('b').tail
mi.publisher = unicode(self.repub.sub('', pub).strip())
d = self.repub.search(pub)
if d is not None:
d = d.group(1)
try:
default = utcnow().replace(day=15)
if self.lang != 'all':
d = replace_months(d, self.lang)
d = parse_date(d, assume_utc=True, default=default)
mi.pubdate = d
except:
report(verbose)
#ISBN
elt = filter(lambda x: self.reisbn.search(x.find('b').text), elts)
if elt:
isbn = elt[0].find('b').tail.replace('-', '').strip()
if check_isbn(isbn):
mi.isbn = unicode(isbn)
elif len(elt) > 1:
isbn = elt[1].find('b').tail.replace('-', '').strip()
if check_isbn(isbn):
mi.isbn = unicode(isbn)
#Langue
elt = filter(lambda x: self.relang.search(x.find('b').text), elts)
if elt:
langue = elt[0].find('b').tail.strip()
if langue:
mi.language = unicode(langue)
#ratings
elt = filter(lambda x: self.reratelt.search(x.find('b').text), elts)
if elt:
ratings = elt[0].find_class('swSprite')
if ratings:
ratings = self.rerat.findall(ratings[0].get('title'))
if len(ratings) == 2:
mi.rating = float(ratings[0])/float(ratings[1]) * 5
return mi
def fill_MI(self, entry, title, authors, browser, verbose):
mi = MetaInformation(title, authors)
mi.author_sort = authors_to_sort_string(authors)
mi.comments = self.get_description(entry, verbose)
mi = self.get_book_info(entry, mi, verbose)
mi.tags = self.get_tags(entry, browser, verbose)
return mi
def get_individual_metadata(self, browser, linkdata, verbose):
try:
raw = browser.open_novisit(linkdata).read()
except Exception, e:
report(verbose)
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return
raise
if '<title>404 - ' in raw:
report(verbose)
return
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
try:
return soupparser.fromstring(raw)
except:
try:
#remove ASCII invalid chars
return soupparser.fromstring(clean_ascii_chars(raw))
except:
report(verbose)
return
def populate(self, entries, browser, verbose=False):
for x in entries:
try:
entry = self.get_individual_metadata(browser, x, verbose)
# clean results
# inv_ids = ('divsinglecolumnminwidth', 'sims.purchase', 'AutoBuyXGetY', 'A9AdsMiddleBoxTop')
# inv_class = ('buyingDetailsGrid', 'productImageGrid')
# inv_tags ={'script': True, 'style': True, 'form': False}
# self.clean_entry(entry, invalid_id=inv_ids)
title = self.get_title(entry)
authors = self.get_authors(entry)
except Exception, e:
if verbose:
print 'Failed to get all details for an entry'
print e
print 'URL who failed:', x
report(verbose)
continue
self.append(self.fill_MI(entry, title, authors, browser, verbose))
def search(title=None, author=None, publisher=None, isbn=None,
max_results=5, verbose=False, keywords=None, lang='all'):
br = browser()
entries, baseurl = Query(title=title, author=author, isbn=isbn, publisher=publisher,
keywords=keywords, max_results=max_results,rlang=lang)(br, verbose)
if entries is None or len(entries) == 0:
return
#List of entry
ans = ResultList(baseurl, lang)
ans.populate(entries, br, verbose)
return ans
def option_parser():
parser = OptionParser(textwrap.dedent(\
_('''\
%prog [options]
Fetch book metadata from Amazon. You must specify one of title, author,
ISBN, publisher or keywords. Will fetch a maximum of 10 matches,
so you should make your query as specific as possible.
You can chose the language for metadata retrieval:
All & english & french & german & spanish
'''
)))
parser.add_option('-t', '--title', help='Book title')
parser.add_option('-a', '--author', help='Book author(s)')
parser.add_option('-p', '--publisher', help='Book publisher')
parser.add_option('-i', '--isbn', help='Book ISBN')
parser.add_option('-k', '--keywords', help='Keywords')
parser.add_option('-m', '--max-results', default=10,
help='Maximum number of results to fetch')
parser.add_option('-l', '--lang', default='all',
help='Chosen language for metadata search (all, en, fr, es, de)')
parser.add_option('-v', '--verbose', default=0, action='count',
help='Be more verbose about errors')
return parser
def main(args=sys.argv):
parser = option_parser()
opts, args = parser.parse_args(args)
try:
results = search(opts.title, opts.author, isbn=opts.isbn, publisher=opts.publisher,
keywords=opts.keywords, verbose=opts.verbose, max_results=opts.max_results,
lang=opts.lang)
except AssertionError:
report(True)
parser.print_help()
return 1
if results is None or len(results) == 0:
print 'No result found for this search!'
return 0
for result in results:
print unicode(result).encode(preferred_encoding, 'replace')
print
if __name__ == '__main__':
sys.exit(main())

2
src/calibre/ebooks/metadata/book/base.py
View File

@ -531,6 +531,8 @@ class Metadata(object):
res = format_date(res, cmeta['display'].get('date_format','dd MMM yyyy'))
elif datatype == 'bool':
res = _('Yes') if res else _('No')
elif datatype == 'rating':
res = res/2
return (name, unicode(res), orig_res, cmeta)
# Translate aliases into the standard field name

390
src/calibre/ebooks/metadata/fictionwise.py Normal file
View File

@ -0,0 +1,390 @@
from __future__ import with_statement
__license__ = 'GPL 3'
__copyright__ = '2010, sengian <sengian1@gmail.com>'
__docformat__ = 'restructuredtext en'
import sys, textwrap, re, traceback, socket
from urllib import urlencode
from lxml.html import soupparser, tostring
from calibre import browser, preferred_encoding
from calibre.ebooks.chardet import xml_to_unicode
from calibre.ebooks.metadata import MetaInformation, check_isbn, \
authors_to_sort_string
from calibre.library.comments import sanitize_comments_html
from calibre.ebooks.metadata.fetch import MetadataSource
from calibre.utils.config import OptionParser
from calibre.utils.date import parse_date, utcnow
from calibre.utils.cleantext import clean_ascii_chars
class Fictionwise(MetadataSource): # {{{
author = 'Sengian'
name = 'Fictionwise'
description = _('Downloads metadata from Fictionwise')
has_html_comments = True
def fetch(self):
try:
self.results = search(self.title, self.book_author, self.publisher,
self.isbn, max_results=10, verbose=self.verbose)
except Exception, e:
self.exception = e
self.tb = traceback.format_exc()
# }}}
class FictionwiseError(Exception):
pass
def report(verbose):
if verbose:
traceback.print_exc()
class Query(object):
BASE_URL = 'http://www.fictionwise.com/servlet/mw'
def __init__(self, title=None, author=None, publisher=None, keywords=None, max_results=20):
assert not(title is None and author is None and publisher is None and keywords is None)
assert (max_results < 21)
self.max_results = int(max_results)
q = { 'template' : 'searchresults_adv.htm' ,
'searchtitle' : '',
'searchauthor' : '',
'searchpublisher' : '',
'searchkeyword' : '',
#possibilities startoflast, fullname, lastfirst
'searchauthortype' : 'startoflast',
'searchcategory' : '',
'searchcategory2' : '',
'searchprice_s' : '0',
'searchprice_e' : 'ANY',
'searchformat' : '',
'searchgeo' : 'US',
'searchfwdatetype' : '',
#maybe use dates fields if needed?
#'sortorder' : 'DESC',
#many options available: b.SortTitle, a.SortName,
#b.DateFirstPublished, b.FWPublishDate
'sortby' : 'b.SortTitle'
}
if title is not None:
q['searchtitle'] = title
if author is not None:
q['searchauthor'] = author
if publisher is not None:
q['searchpublisher'] = publisher
if keywords is not None:
q['searchkeyword'] = keywords
if isinstance(q, unicode):
q = q.encode('utf-8')
self.urldata = urlencode(q)
def __call__(self, browser, verbose, timeout = 5.):
if verbose:
print _('Query: %s') % self.BASE_URL+self.urldata
try:
raw = browser.open_novisit(self.BASE_URL, self.urldata, timeout=timeout).read()
except Exception, e:
report(verbose)
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return
if isinstance(getattr(e, 'args', [None])[0], socket.timeout):
raise FictionwiseError(_('Fictionwise timed out. Try again later.'))
raise FictionwiseError(_('Fictionwise encountered an error.'))
if '<title>404 - ' in raw:
return
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
try:
feed = soupparser.fromstring(raw)
except:
try:
#remove ASCII invalid chars
feed = soupparser.fromstring(clean_ascii_chars(raw))
except:
return None
# get list of results as links
results = feed.xpath("//table[3]/tr/td[2]/table/tr/td/p/table[2]/tr[@valign]")
results = results[:self.max_results]
results = [i.xpath('descendant-or-self::a')[0].get('href') for i in results]
#return feed if no links ie normally a single book or nothing
if not results:
results = [feed]
return results
class ResultList(list):
BASE_URL = 'http://www.fictionwise.com'
COLOR_VALUES = {'BLUE': 4, 'GREEN': 3, 'YELLOW': 2, 'RED': 1, 'NA': 0}
def __init__(self):
self.retitle = re.compile(r'\[[^\[\]]+\]')
self.rechkauth = re.compile(r'.*book\s*by', re.I)
self.redesc = re.compile(r'book\s*description\s*:\s*(<br[^>]+>)*(?P<desc>.*)<br[^>]*>.{,15}publisher\s*:', re.I)
self.repub = re.compile(r'.*publisher\s*:\s*', re.I)
self.redate = re.compile(r'.*release\s*date\s*:\s*', re.I)
self.retag = re.compile(r'.*book\s*category\s*:\s*', re.I)
self.resplitbr = re.compile(r'<br[^>]*>', re.I)
self.recomment = re.compile(r'(?s)<!--.*?-->')
self.reimg = re.compile(r'<img[^>]*>', re.I)
self.resanitize = re.compile(r'\[HTML_REMOVED\]\s*', re.I)
self.renbcom = re.compile('(?P<nbcom>\d+)\s*Reader Ratings:')
self.recolor = re.compile('(?P<ncolor>[^/]+).gif')
self.resplitbrdiv = re.compile(r'(<br[^>]+>|</?div[^>]*>)', re.I)
self.reisbn = re.compile(r'.*ISBN\s*:\s*', re.I)
def strip_tags_etree(self, etreeobj, invalid_tags):
for (itag, rmv) in invalid_tags.iteritems():
if rmv:
for elts in etreeobj.getiterator(itag):
elts.drop_tree()
else:
for elts in etreeobj.getiterator(itag):
elts.drop_tag()
def clean_entry(self, entry, invalid_tags = {'script': True},
invalid_id = (), invalid_class=(), invalid_xpath = ()):
#invalid_tags: remove tag and keep content if False else remove
#remove tags
if invalid_tags:
self.strip_tags_etree(entry, invalid_tags)
#remove xpath
if invalid_xpath:
for eltid in invalid_xpath:
elt = entry.xpath(eltid)
for el in elt:
el.drop_tree()
#remove id
if invalid_id:
for eltid in invalid_id:
elt = entry.get_element_by_id(eltid)
if elt is not None:
elt.drop_tree()
#remove class
if invalid_class:
for eltclass in invalid_class:
elts = entry.find_class(eltclass)
if elts is not None:
for elt in elts:
elt.drop_tree()
def output_entry(self, entry, prettyout = True, htmlrm="\d+"):
out = tostring(entry, pretty_print=prettyout)
#try to work around tostring to remove this encoding for exemle
reclean = re.compile('(\n+|\t+|\r+|&#'+htmlrm+';)')
return reclean.sub('', out)
def get_title(self, entry):
title = entry.findtext('./')
return self.retitle.sub('', title).strip()
def get_authors(self, entry):
authortext = entry.find('./br').tail
if not self.rechkauth.search(authortext):
return []
authortext = self.rechkauth.sub('', authortext)
return [a.strip() for a in authortext.split('&')]
def get_rating(self, entrytable, verbose):
nbcomment = tostring(entrytable.getprevious())
try:
nbcomment = self.renbcom.search(nbcomment).group("nbcom")
except:
report(verbose)
return None
hval = dict((self.COLOR_VALUES[self.recolor.search(image.get('src', default='NA.gif')).group("ncolor")],
float(image.get('height', default=0))) \
for image in entrytable.getiterator('img'))
#ratings as x/5
return float(1.25*sum(k*v for (k, v) in hval.iteritems())/sum(hval.itervalues()))
def get_description(self, entry):
description = self.output_entry(entry.xpath('./p')[1],htmlrm="")
description = self.redesc.search(description)
if not description or not description.group("desc"):
return None
#remove invalid tags
description = self.reimg.sub('', description.group("desc"))
description = self.recomment.sub('', description)
description = self.resanitize.sub('', sanitize_comments_html(description))
return _('SUMMARY:\n %s') % re.sub(r'\n\s+</p>','\n</p>', description)
def get_publisher(self, entry):
publisher = self.output_entry(entry.xpath('./p')[1])
publisher = filter(lambda x: self.repub.search(x) is not None,
self.resplitbr.split(publisher))
if not len(publisher):
return None
publisher = self.repub.sub('', publisher[0])
return publisher.split(',')[0].strip()
def get_tags(self, entry):
tag = self.output_entry(entry.xpath('./p')[1])
tag = filter(lambda x: self.retag.search(x) is not None,
self.resplitbr.split(tag))
if not len(tag):
return []
return map(lambda x: x.strip(), self.retag.sub('', tag[0]).split('/'))
def get_date(self, entry, verbose):
date = self.output_entry(entry.xpath('./p')[1])
date = filter(lambda x: self.redate.search(x) is not None,
self.resplitbr.split(date))
if not len(date):
return None
try:
d = self.redate.sub('', date[0])
if d:
default = utcnow().replace(day=15)
d = parse_date(d, assume_utc=True, default=default)
else:
d = None
except:
report(verbose)
d = None
return d
def get_ISBN(self, entry):
isbns = self.output_entry(entry.xpath('./p')[2])
isbns = filter(lambda x: self.reisbn.search(x) is not None,
self.resplitbrdiv.split(isbns))
if not len(isbns):
return None
isbns = [self.reisbn.sub('', x) for x in isbns if check_isbn(self.reisbn.sub('', x))]
return sorted(isbns, cmp=lambda x,y:cmp(len(x), len(y)))[-1]
def fill_MI(self, entry, title, authors, ratings, verbose):
mi = MetaInformation(title, authors)
mi.rating = ratings
mi.comments = self.get_description(entry)
mi.publisher = self.get_publisher(entry)
mi.tags = self.get_tags(entry)
mi.pubdate = self.get_date(entry, verbose)
mi.isbn = self.get_ISBN(entry)
mi.author_sort = authors_to_sort_string(authors)
return mi
def get_individual_metadata(self, browser, linkdata, verbose):
try:
raw = browser.open_novisit(self.BASE_URL + linkdata).read()
except Exception, e:
report(verbose)
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return
if isinstance(getattr(e, 'args', [None])[0], socket.timeout):
raise FictionwiseError(_('Fictionwise timed out. Try again later.'))
raise FictionwiseError(_('Fictionwise encountered an error.'))
if '<title>404 - ' in raw:
report(verbose)
return
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
try:
return soupparser.fromstring(raw)
except:
try:
#remove ASCII invalid chars
return soupparser.fromstring(clean_ascii_chars(raw))
except:
return None
def populate(self, entries, browser, verbose=False):
inv_tags ={'script': True, 'a': False, 'font': False, 'strong': False, 'b': False,
'ul': False, 'span': False}
inv_xpath =('./table',)
#single entry
if len(entries) == 1 and not isinstance(entries[0], str):
try:
entry = entries.xpath("//table[3]/tr/td[2]/table[1]/tr/td/font/table/tr/td")
self.clean_entry(entry, invalid_tags=inv_tags, invalid_xpath=inv_xpath)
title = self.get_title(entry)
#maybe strenghten the search
ratings = self.get_rating(entry.xpath("./p/table")[1], verbose)
authors = self.get_authors(entry)
except Exception, e:
if verbose:
print _('Failed to get all details for an entry')
print e
return
self.append(self.fill_MI(entry, title, authors, ratings, verbose))
else:
#multiple entries
for x in entries:
try:
entry = self.get_individual_metadata(browser, x, verbose)
entry = entry.xpath("//table[3]/tr/td[2]/table[1]/tr/td/font/table/tr/td")[0]
self.clean_entry(entry, invalid_tags=inv_tags, invalid_xpath=inv_xpath)
title = self.get_title(entry)
#maybe strenghten the search
ratings = self.get_rating(entry.xpath("./p/table")[1], verbose)
authors = self.get_authors(entry)
except Exception, e:
if verbose:
print _('Failed to get all details for an entry')
print e
continue
self.append(self.fill_MI(entry, title, authors, ratings, verbose))
def search(title=None, author=None, publisher=None, isbn=None,
min_viewability='none', verbose=False, max_results=5,
keywords=None):
br = browser()
entries = Query(title=title, author=author, publisher=publisher,
keywords=keywords, max_results=max_results)(br, verbose, timeout = 15.)
#List of entry
ans = ResultList()
ans.populate(entries, br, verbose)
return ans
def option_parser():
parser = OptionParser(textwrap.dedent(\
_('''\
%prog [options]
Fetch book metadata from Fictionwise. You must specify one of title, author,
or keywords. No ISBN specification possible. Will fetch a maximum of 20 matches,
so you should make your query as specific as possible.
''')
))
parser.add_option('-t', '--title', help=_('Book title'))
parser.add_option('-a', '--author', help=_('Book author(s)'))
parser.add_option('-p', '--publisher', help=_('Book publisher'))
parser.add_option('-k', '--keywords', help=_('Keywords'))
parser.add_option('-m', '--max-results', default=20,
help=_('Maximum number of results to fetch'))
parser.add_option('-v', '--verbose', default=0, action='count',
help=_('Be more verbose about errors'))
return parser
def main(args=sys.argv):
parser = option_parser()
opts, args = parser.parse_args(args)
try:
results = search(opts.title, opts.author, publisher=opts.publisher,
keywords=opts.keywords, verbose=opts.verbose, max_results=opts.max_results)
except AssertionError:
report(True)
parser.print_help()
return 1
if results is None or len(results) == 0:
print _('No result found for this search!')
return 0
for result in results:
print unicode(result).encode(preferred_encoding, 'replace')
print
if __name__ == '__main__':
sys.exit(main())

113
src/calibre/ebooks/metadata/nicebooks.py
View File

@ -10,7 +10,8 @@ from copy import deepcopy
from lxml.html import soupparser
from calibre.utils.date import parse_date, utcnow
from calibre.utils.date import parse_date, utcnow, replace_months
from calibre.utils.cleantext import clean_ascii_chars
from calibre import browser, preferred_encoding
from calibre.ebooks.chardet import xml_to_unicode
from calibre.ebooks.metadata import MetaInformation, check_isbn, \
@ -71,31 +72,16 @@ class NiceBooksCovers(CoverDownload):
traceback.format_exc(), self.name))
class NiceBooksError(Exception):
pass
class ISBNNotFound(NiceBooksError):
pass
def report(verbose):
if verbose:
import traceback
traceback.print_exc()
def replace_monthsfr(datefr):
# Replace french months by english equivalent for parse_date
frtoen = {
u'[jJ]anvier': u'jan',
u'[fF].vrier': u'feb',
u'[mM]ars': u'mar',
u'[aA]vril': u'apr',
u'[mM]ai': u'may',
u'[jJ]uin': u'jun',
u'[jJ]uillet': u'jul',
u'[aA]o.t': u'aug',
u'[sS]eptembre': u'sep',
u'[Oo]ctobre': u'oct',
u'[nN]ovembre': u'nov',
u'[dD].cembre': u'dec' }
for k in frtoen.iterkeys():
tmp = re.sub(k, frtoen[k], datefr)
if tmp <> datefr: break
return tmp
class Query(object):
BASE_URL = 'http://fr.nicebooks.com/'
@ -119,7 +105,7 @@ class Query(object):
def __call__(self, browser, verbose, timeout = 5.):
if verbose:
print 'Query:', self.BASE_URL+self.urldata
print _('Query: %s') % self.BASE_URL+self.urldata
try:
raw = browser.open_novisit(self.BASE_URL+self.urldata, timeout=timeout).read()
@ -128,7 +114,9 @@ class Query(object):
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return
raise
if isinstance(getattr(e, 'args', [None])[0], socket.timeout):
raise NiceBooksError(_('Nicebooks timed out. Try again later.'))
raise NiceBooksError(_('Nicebooks encountered an error.'))
if '<title>404 - ' in raw:
return
raw = xml_to_unicode(raw, strip_encoding_pats=True,
@ -136,7 +124,11 @@ class Query(object):
try:
feed = soupparser.fromstring(raw)
except:
return
try:
#remove ASCII invalid chars
feed = soupparser.fromstring(clean_ascii_chars(raw))
except:
return None
#nb of page to call
try:
@ -161,7 +153,11 @@ class Query(object):
try:
feed = soupparser.fromstring(raw)
except:
continue
try:
#remove ASCII invalid chars
feed = soupparser.fromstring(clean_ascii_chars(raw))
except:
continue
pages.append(feed)
results = []
@ -180,14 +176,12 @@ class ResultList(list):
self.reautclean = re.compile(u'\s*\(.*\)\s*')
def get_title(self, entry):
# title = deepcopy(entry.find("div[@id='book-info']"))
title = deepcopy(entry)
title.remove(title.find("dl[@title='Informations sur le livre']"))
title = ' '.join([i.text_content() for i in title.iterchildren()])
return unicode(title.replace('\n', ''))
def get_authors(self, entry):
# author = entry.find("div[@id='book-info']/dl[@title='Informations sur le livre']")
author = entry.find("dl[@title='Informations sur le livre']")
authortext = []
for x in author.getiterator('dt'):
@ -223,7 +217,7 @@ class ResultList(list):
d = x.getnext().text_content()
try:
default = utcnow().replace(day=15)
d = replace_monthsfr(d)
d = replace_months(d, 'fr')
d = parse_date(d, assume_utc=True, default=default)
mi.pubdate = d
except:
@ -234,11 +228,6 @@ class ResultList(list):
mi = MetaInformation(title, authors)
mi.author_sort = authors_to_sort_string(authors)
mi.comments = self.get_description(entry, verbose)
# entry = entry.find("dl[@title='Informations sur le livre']")
# mi.publisher = self.get_publisher(entry)
# mi.pubdate = self.get_date(entry, verbose)
# mi.isbn = self.get_ISBN(entry)
# mi.language = self.get_language(entry)
return self.get_book_info(entry, mi, verbose)
def get_individual_metadata(self, browser, linkdata, verbose):
@ -249,7 +238,9 @@ class ResultList(list):
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return
raise
if isinstance(getattr(e, 'args', [None])[0], socket.timeout):
raise NiceBooksError(_('Nicebooks timed out. Try again later.'))
raise NiceBooksError(_('Nicebooks encountered an error.'))
if '<title>404 - ' in raw:
report(verbose)
return
@ -258,7 +249,11 @@ class ResultList(list):
try:
feed = soupparser.fromstring(raw)
except:
return
try:
#remove ASCII invalid chars
feed = soupparser.fromstring(clean_ascii_chars(raw))
except:
return None
# get results
return feed.xpath("//div[@id='container']")[0]
@ -292,13 +287,6 @@ class ResultList(list):
continue
self.append(self.fill_MI(entry, title, authors, verbose))
class NiceBooksError(Exception):
pass
class ISBNNotFound(NiceBooksError):
pass
class Covers(object):
def __init__(self, isbn = None):
@ -329,11 +317,10 @@ class Covers(object):
return cover, ext if ext else 'jpg'
except Exception, err:
if isinstance(getattr(err, 'args', [None])[0], socket.timeout):
err = NiceBooksError(_('Nicebooks timed out. Try again later.'))
raise err
raise NiceBooksError(_('Nicebooks timed out. Try again later.'))
if not len(self.urlimg):
if not self.isbnf:
raise ISBNNotFound('ISBN: '+self.isbn+_(' not found.'))
raise ISBNNotFound(_('ISBN: %s not found.') % self.isbn)
raise NiceBooksError(_('An errror occured with Nicebooks cover fetcher'))
@ -341,10 +328,10 @@ def search(title=None, author=None, publisher=None, isbn=None,
max_results=5, verbose=False, keywords=None):
br = browser()
entries = Query(title=title, author=author, isbn=isbn, publisher=publisher,
keywords=keywords, max_results=max_results)(br, verbose)
keywords=keywords, max_results=max_results)(br, verbose,timeout = 10.)
if entries is None or len(entries) == 0:
return
return None
#List of entry
ans = ResultList()
@ -364,28 +351,28 @@ def cover_from_isbn(isbn, timeout = 5.):
def option_parser():
parser = OptionParser(textwrap.dedent(\
'''\
_('''\
%prog [options]
Fetch book metadata from Nicebooks. You must specify one of title, author,
ISBN, publisher or keywords. Will fetch a maximum of 20 matches,
so you should make your query as specific as possible.
It can also get covers if the option is activated.
'''
''')
))
parser.add_option('-t', '--title', help='Book title')
parser.add_option('-a', '--author', help='Book author(s)')
parser.add_option('-p', '--publisher', help='Book publisher')
parser.add_option('-i', '--isbn', help='Book ISBN')
parser.add_option('-k', '--keywords', help='Keywords')
parser.add_option('-t', '--title', help=_('Book title'))
parser.add_option('-a', '--author', help=_('Book author(s)'))
parser.add_option('-p', '--publisher', help=_('Book publisher'))
parser.add_option('-i', '--isbn', help=_('Book ISBN'))
parser.add_option('-k', '--keywords', help=_('Keywords'))
parser.add_option('-c', '--covers', default=0,
help='Covers: 1-Check/ 2-Download')
help=_('Covers: 1-Check/ 2-Download'))
parser.add_option('-p', '--coverspath', default='',
help='Covers files path')
help=_('Covers files path'))
parser.add_option('-m', '--max-results', default=20,
help='Maximum number of results to fetch')
help=_('Maximum number of results to fetch'))
parser.add_option('-v', '--verbose', default=0, action='count',
help='Be more verbose about errors')
help=_('Be more verbose about errors'))
return parser
def main(args=sys.argv):
@ -400,15 +387,15 @@ def main(args=sys.argv):
parser.print_help()
return 1
if results is None or len(results) == 0:
print 'No result found for this search!'
print _('No result found for this search!')
return 0
for result in results:
print unicode(result).encode(preferred_encoding, 'replace')
covact = int(opts.covers)
if covact == 1:
textcover = 'No cover found!'
textcover = _('No cover found!')
if check_for_cover(result.isbn):
textcover = 'A cover was found for this book'
textcover = _('A cover was found for this book')
print textcover
elif covact == 2:
cover_data, ext = cover_from_isbn(result.isbn)
@ -417,7 +404,7 @@ def main(args=sys.argv):
cpath = os.path.normpath(opts.coverspath + '/' + result.isbn)
oname = os.path.abspath(cpath+'.'+ext)
open(oname, 'wb').write(cover_data)
print 'Cover saved to file ', oname
print _('Cover saved to file '), oname
print
if __name__ == '__main__':

91
src/calibre/ebooks/metadata/worker.py
View File

@ -8,12 +8,12 @@ __docformat__ = 'restructuredtext en'
from threading import Thread
from Queue import Empty
import os, time, sys, shutil
import os, time, sys, shutil, json
from calibre.utils.ipc.job import ParallelJob
from calibre.utils.ipc.server import Server
from calibre.ptempfile import PersistentTemporaryDirectory, TemporaryDirectory
from calibre import prints
from calibre import prints, isbytestring
from calibre.constants import filesystem_encoding
@ -194,14 +194,42 @@ class SaveWorker(Thread):
self.daemon = True
self.path, self.opts = path, opts
self.ids = ids
self.library_path = db.library_path
self.db = db
self.canceled = False
self.result_queue = result_queue
self.error = None
self.spare_server = spare_server
self.start()
def collect_data(self, ids):
from calibre.ebooks.metadata.opf2 import metadata_to_opf
data = {}
for i in set(ids):
mi = self.db.get_metadata(i, index_is_id=True, get_cover=True)
opf = metadata_to_opf(mi)
if isbytestring(opf):
opf = opf.decode('utf-8')
cpath = None
if mi.cover:
cpath = mi.cover
if isbytestring(cpath):
cpath = cpath.decode(filesystem_encoding)
formats = {}
if mi.formats:
for fmt in mi.formats:
fpath = self.db.format_abspath(i, fmt, index_is_id=True)
if fpath is not None:
if isbytestring(fpath):
fpath = fpath.decode(filesystem_encoding)
formats[fmt.lower()] = fpath
data[i] = [opf, cpath, formats]
return data
def run(self):
with TemporaryDirectory('save_to_disk_data') as tdir:
self._run(tdir)
def _run(self, tdir):
from calibre.library.save_to_disk import config
server = Server() if self.spare_server is None else self.spare_server
ids = set(self.ids)
@ -212,12 +240,19 @@ class SaveWorker(Thread):
for pref in c.preferences:
recs[pref.name] = getattr(self.opts, pref.name)
plugboards = self.db.prefs.get('plugboards', {})
for i, task in enumerate(tasks):
tids = [x[-1] for x in task]
data = self.collect_data(tids)
dpath = os.path.join(tdir, '%d.json'%i)
with open(dpath, 'wb') as f:
f.write(json.dumps(data, ensure_ascii=False).encode('utf-8'))
job = ParallelJob('save_book',
'Save books (%d of %d)'%(i, len(tasks)),
lambda x,y:x,
args=[tids, self.library_path, self.path, recs])
args=[tids, dpath, plugboards, self.path, recs])
jobs.add(job)
server.add_job(job)
@ -226,21 +261,21 @@ class SaveWorker(Thread):
time.sleep(0.2)
running = False
for job in jobs:
job.update(consume_notifications=False)
while True:
try:
id, title, ok, tb = job.notifications.get_nowait()[0]
if id in ids:
self.result_queue.put((id, title, ok, tb))
ids.remove(id)
except Empty:
break
self.get_notifications(job, ids)
if not job.is_finished:
running = True
if not running:
break
for job in jobs:
if not job.result:
continue
for id_, title, ok, tb in job.result:
if id_ in ids:
self.result_queue.put((id_, title, ok, tb))
ids.remove(id_)
server.close()
time.sleep(1)
@ -257,21 +292,39 @@ class SaveWorker(Thread):
except:
pass
def get_notifications(self, job, ids):
job.update(consume_notifications=False)
while True:
try:
id, title, ok, tb = job.notifications.get_nowait()[0]
if id in ids:
self.result_queue.put((id, title, ok, tb))
ids.remove(id)
except Empty:
break
def save_book(task, library_path, path, recs, notification=lambda x,y:x):
from calibre.library.database2 import LibraryDatabase2
db = LibraryDatabase2(library_path)
from calibre.library.save_to_disk import config, save_to_disk
def save_book(ids, dpath, plugboards, path, recs, notification=lambda x,y:x):
from calibre.library.save_to_disk import config, save_serialized_to_disk
from calibre.customize.ui import apply_null_metadata
opts = config().parse()
for name in recs:
setattr(opts, name, recs[name])
results = []
def callback(id, title, failed, tb):
results.append((id, title, not failed, tb))
notification((id, title, not failed, tb))
return True
with apply_null_metadata:
save_to_disk(db, task, path, opts, callback)
data_ = json.loads(open(dpath, 'rb').read().decode('utf-8'))
data = {}
for k, v in data_.iteritems():
data[int(k)] = v
with apply_null_metadata:
save_serialized_to_disk(ids, data, plugboards, path, opts, callback)
return results

41
src/calibre/ebooks/mobi/mobiml.py
View File

@ -10,9 +10,10 @@ import copy
import re
from lxml import etree
from calibre.ebooks.oeb.base import namespace, barename
from calibre.ebooks.oeb.base import XHTML, XHTML_NS, OEB_DOCS
from calibre.ebooks.oeb.base import XHTML, XHTML_NS, OEB_DOCS, urlnormalize
from calibre.ebooks.oeb.stylizer import Stylizer
from calibre.ebooks.oeb.transforms.flatcss import KeyMapper
from calibre.utils.magick.draw import identify_data
MBP_NS = 'http://mobipocket.com/ns/mbp'
def MBP(name): return '{%s}%s' % (MBP_NS, name)
@ -121,6 +122,7 @@ class MobiMLizer(object):
body = item.data.find(XHTML('body'))
nroot = etree.Element(XHTML('html'), nsmap=MOBI_NSMAP)
nbody = etree.SubElement(nroot, XHTML('body'))
self.current_spine_item = item
self.mobimlize_elem(body, stylizer, BlockState(nbody),
[FormatState()])
item.data = nroot
@ -357,8 +359,9 @@ class MobiMLizer(object):
if tag == 'img' and 'src' in elem.attrib:
istate.attrib['src'] = elem.attrib['src']
istate.attrib['align'] = 'baseline'
cssdict = style.cssdict()
for prop in ('width', 'height'):
if style[prop] != 'auto':
if cssdict[prop] != 'auto':
value = style[prop]
if value == getattr(self.profile, prop):
result = '100%'
@ -371,8 +374,40 @@ class MobiMLizer(object):
(72./self.profile.dpi)))
except:
continue
result = "%d"%pixs
result = str(pixs)
istate.attrib[prop] = result
if 'width' not in istate.attrib or 'height' not in istate.attrib:
href = self.current_spine_item.abshref(elem.attrib['src'])
try:
item = self.oeb.manifest.hrefs[urlnormalize(href)]
except:
self.oeb.logger.warn('Failed to find image:',
href)
else:
try:
width, height = identify_data(item.data)[:2]
except:
self.oeb.logger.warn('Invalid image:', href)
else:
if 'width' not in istate.attrib and 'height' not in \
istate.attrib:
istate.attrib['width'] = str(width)
istate.attrib['height'] = str(height)
else:
ar = float(width)/float(height)
if 'width' not in istate.attrib:
try:
width = int(istate.attrib['height'])*ar
except:
pass
istate.attrib['width'] = str(int(width))
else:
try:
height = int(istate.attrib['width'])/ar
except:
pass
istate.attrib['height'] = str(int(height))
item.unload_data_from_memory()
elif tag == 'hr' and asfloat(style['width']) > 0:
prop = style['width'] / self.profile.width
istate.attrib['width'] = "%d%%" % int(round(prop * 100))

5
src/calibre/ebooks/oeb/base.py
View File

@ -607,7 +607,7 @@ class Metadata(object):
key = barename(key)
attrib[key] = prefixname(value, nsrmap)
if namespace(self.term) == DC11_NS:
name = DC(barename(self.term).title())
name = DC(icu_title(barename(self.term)))
elem = element(dcmeta, name, attrib=attrib)
elem.text = self.value
else:
@ -775,6 +775,7 @@ class Manifest(object):
return u'Item(id=%r, href=%r, media_type=%r)' \
% (self.id, self.href, self.media_type)
# Parsing {{{
def _parse_xml(self, data):
data = xml_to_unicode(data, strip_encoding_pats=True,
assume_utf8=True, resolve_entities=True)[0]
@ -1035,6 +1036,8 @@ class Manifest(object):
data = item.data.cssText
return ('utf-8', data)
# }}}
@dynamic_property
def data(self):
doc = """Provides MIME type sensitive access to the manifest

5
src/calibre/ebooks/oeb/iterator.py
View File

@ -96,7 +96,10 @@ class EbookIterator(object):
def search(self, text, index, backwards=False):
text = text.lower()
for i, path in enumerate(self.spine):
pmap = [(i, path) for i, path in enumerate(self.spine)]
if backwards:
pmap.reverse()
for i, path in pmap:
if (backwards and i < index) or (not backwards and i > index):
if text in open(path, 'rb').read().decode(path.encoding).lower():
return i

2
src/calibre/ebooks/oeb/reader.py
View File

@ -544,7 +544,7 @@ class OEBReader(object):
data = render_html_svg_workaround(path, self.logger)
if not data:
data = ''
id, href = self.oeb.manifest.generate('cover', 'cover.jpeg')
id, href = self.oeb.manifest.generate('cover', 'cover.jpg')
item = self.oeb.manifest.add(id, href, JPEG_MIME, data=data)
return item

23
src/calibre/ebooks/oeb/stylizer.py
View File

@ -253,7 +253,10 @@ class Stylizer(object):
upd = {}
for prop in ('width', 'height'):
val = elem.get(prop, '').strip()
del elem.attrib[prop]
try:
del elem.attrib[prop]
except:
pass
if val:
if num_pat.match(val) is not None:
val += 'px'
@ -572,7 +575,7 @@ class Style(object):
if parent is not None:
base = parent.width
else:
base = self._profile.width
base = self._profile.width_pts
if 'width' in self._element.attrib:
width = self._element.attrib['width']
elif 'width' in self._style:
@ -584,6 +587,13 @@ class Style(object):
if isinstance(result, (unicode, str, bytes)):
result = self._profile.width
self._width = result
if 'max-width' in self._style:
result = self._unit_convert(self._style['max-width'], base=base)
if isinstance(result, (unicode, str, bytes)):
result = self._width
if result < self._width:
self._width = result
return self._width
@property
@ -595,7 +605,7 @@ class Style(object):
if parent is not None:
base = parent.height
else:
base = self._profile.height
base = self._profile.height_pts
if 'height' in self._element.attrib:
height = self._element.attrib['height']
elif 'height' in self._style:
@ -607,6 +617,13 @@ class Style(object):
if isinstance(result, (unicode, str, bytes)):
result = self._profile.height
self._height = result
if 'max-height' in self._style:
result = self._unit_convert(self._style['max-height'], base=base)
if isinstance(result, (unicode, str, bytes)):
result = self._height
if result < self._height:
self._height = result
return self._height
@property

130
src/calibre/ebooks/oeb/transforms/filenames.py Normal file
View File

@ -0,0 +1,130 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
__license__ = 'GPL v3'
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import posixpath
from urlparse import urldefrag
from lxml import etree
import cssutils
from calibre.ebooks.oeb.base import rewrite_links, urlnormalize
class RenameFiles(object):
'''
Rename files and adjust all links pointing to them. Note that the spine
and manifest are not touched by this transform.
'''
def __init__(self, rename_map):
self.rename_map = rename_map
def __call__(self, oeb, opts):
self.log = oeb.logger
self.opts = opts
self.oeb = oeb
for item in oeb.manifest.items:
self.current_item = item
if etree.iselement(item.data):
rewrite_links(self.current_item.data, self.url_replacer)
elif hasattr(item.data, 'cssText'):
cssutils.replaceUrls(item.data, self.url_replacer)
if self.oeb.guide:
for ref in self.oeb.guide.values():
href = urlnormalize(ref.href)
href, frag = urldefrag(href)
replacement = self.rename_map.get(href, None)
if replacement is not None:
nhref = replacement
if frag:
nhref += '#' + frag
ref.href = nhref
if self.oeb.toc:
self.fix_toc_entry(self.oeb.toc)
def fix_toc_entry(self, toc):
if toc.href:
href = urlnormalize(toc.href)
href, frag = urldefrag(href)
replacement = self.rename_map.get(href, None)
if replacement is not None:
nhref = replacement
if frag:
nhref = '#'.join((nhref, frag))
toc.href = nhref
for x in toc:
self.fix_toc_entry(x)
def url_replacer(self, orig_url):
url = urlnormalize(orig_url)
path, frag = urldefrag(url)
href = self.current_item.abshref(path)
replacement = self.rename_map.get(href, None)
if replacement is None:
return orig_url
replacement = self.current_item.relhref(replacement)
if frag:
replacement += '#' + frag
return replacement
class UniqueFilenames(object):
'Ensure that every item in the manifest has a unique filename'
def __call__(self, oeb, opts):
self.log = oeb.logger
self.opts = opts
self.oeb = oeb
self.seen_filenames = set([])
self.rename_map = {}
for item in list(oeb.manifest.items):
fname = posixpath.basename(item.href)
if fname in self.seen_filenames:
suffix = self.unique_suffix(fname)
data = item.data
base, ext = posixpath.splitext(item.href)
nhref = base + suffix + ext
nhref = oeb.manifest.generate(href=nhref)[1]
nitem = oeb.manifest.add(item.id, nhref, item.media_type, data=data,
fallback=item.fallback)
self.seen_filenames.add(posixpath.basename(nhref))
self.rename_map[item.href] = nhref
if item.spine_position is not None:
oeb.spine.insert(item.spine_position, nitem, item.linear)
oeb.spine.remove(item)
oeb.manifest.remove(item)
else:
self.seen_filenames.add(fname)
if self.rename_map:
self.log('Found non-unique filenames, renaming to support broken'
' EPUB readers like FBReader, Aldiko and Stanza...')
from pprint import pformat
self.log.debug(pformat(self.rename_map))
renamer = RenameFiles(self.rename_map)
renamer(oeb, opts)
def unique_suffix(self, fname):
base, ext = posixpath.splitext(fname)
c = 0
while True:
c += 1
suffix = '_u%d'%c
candidate = base + suffix + ext
if candidate not in self.seen_filenames:
return suffix

6
src/calibre/ebooks/oeb/transforms/manglecase.py
View File

@ -50,11 +50,11 @@ class CaseMangler(object):
def text_transform(self, transform, text):
if transform == 'capitalize':
return text.title()
return icu_title(text)
elif transform == 'uppercase':
return text.upper()
return icu_upper(text)
elif transform == 'lowercase':
return text.lower()
return icu_lower(text)
return text
def split_text(self, text):

16
src/calibre/ebooks/pml/output.py
View File

@ -35,6 +35,12 @@ class PMLOutput(OutputFormatPlugin):
OptionRecommendation(name='inline_toc',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Add Table of Contents to beginning of the book.')),
OptionRecommendation(name='full_image_depth',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Do not reduce the size or bit depth of images. Images ' \
'have their size and depth reduced by default to accommodate ' \
'applications that can not convert images on their ' \
'own such as Dropbook.')),
])
def convert(self, oeb_book, output_path, input_plugin, opts, log):
@ -44,16 +50,20 @@ class PMLOutput(OutputFormatPlugin):
with open(os.path.join(tdir, 'index.pml'), 'wb') as out:
out.write(pml.encode(opts.output_encoding, 'replace'))
self.write_images(oeb_book.manifest, pmlmlizer.image_hrefs, tdir)
self.write_images(oeb_book.manifest, pmlmlizer.image_hrefs, tdir, opts)
log.debug('Compressing output...')
pmlz = ZipFile(output_path, 'w')
pmlz.add_dir(tdir)
def write_images(self, manifest, image_hrefs, out_dir):
def write_images(self, manifest, image_hrefs, out_dir, opts):
for item in manifest:
if item.media_type in OEB_RASTER_IMAGES and item.href in image_hrefs.keys():
im = Image.open(cStringIO.StringIO(item.data))
if opts.full_image_depth:
im = Image.open(cStringIO.StringIO(item.data))
else:
im = Image.open(cStringIO.StringIO(item.data)).convert('P')
im.thumbnail((300,300), Image.ANTIALIAS)
data = cStringIO.StringIO()
im.save(data, 'PNG')

63
src/calibre/ebooks/txt/markdownml.py Normal file
View File

@ -0,0 +1,63 @@
# -*- coding: utf-8 -*-
__license__ = 'GPL 3'
__copyright__ = '2009, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
'''
Transform OEB content into Markdown formatted plain text
'''
import re
from lxml import etree
from calibre.utils.html2text import html2text
class MarkdownMLizer(object):
def __init__(self, log):
self.log = log
def extract_content(self, oeb_book, opts):
self.log.info('Converting XHTML to Markdown formatted TXT...')
self.oeb_book = oeb_book
self.opts = opts
return self.mlize_spine()
def mlize_spine(self):
output = [u'']
for item in self.oeb_book.spine:
self.log.debug('Converting %s to Markdown formatted TXT...' % item.href)
html = unicode(etree.tostring(item.data, encoding=unicode))
if not self.opts.keep_links:
html = re.sub(r'<\s*a[^>]*>', '', html)
html = re.sub(r'<\s*/\s*a\s*>', '', html)
if not self.opts.keep_image_references:
html = re.sub(r'<\s*img[^>]*>', '', html)
html = re.sub(r'<\s*img\s*>', '', html)
text = html2text(html)
# Ensure the section ends with at least two new line characters.
# This is to prevent the last paragraph from a section being
# combined into the fist paragraph of the next.
end_chars = text[-4:]
# Convert all newlines to \n
end_chars = end_chars.replace('\r\n', '\n')
end_chars = end_chars.replace('\r', '\n')
end_chars = end_chars[-2:]
if not end_chars[1] == '\n':
text += '\n\n'
if end_chars[1] == '\n' and not end_chars[0] == '\n':
text += '\n'
output += text
output = u''.join(output)
return output

20
src/calibre/ebooks/txt/output.py
View File

@ -8,6 +8,7 @@ import os
from calibre.customize.conversion import OutputFormatPlugin, \
OptionRecommendation
from calibre.ebooks.txt.markdownml import MarkdownMLizer
from calibre.ebooks.txt.txtml import TXTMLizer
from calibre.ebooks.txt.newlines import TxtNewlines, specified_newlines
@ -44,10 +45,27 @@ class TXTOutput(OutputFormatPlugin):
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Force splitting on the max-line-length value when no space '
'is present. Also allows max-line-length to be below the minimum')),
OptionRecommendation(name='markdown_format',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Produce Markdown formatted text.')),
OptionRecommendation(name='keep_links',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Do not remove links within the document. This is only ' \
'useful when paired with the markdown-format option because' \
'links are always removed with plain text output.')),
OptionRecommendation(name='keep_image_references',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Do not remove image references within the document. This is only ' \
'useful when paired with the markdown-format option because' \
'image references are always removed with plain text output.')),
])
def convert(self, oeb_book, output_path, input_plugin, opts, log):
writer = TXTMLizer(log)
if opts.markdown_format:
writer = MarkdownMLizer(log)
else:
writer = TXTMLizer(log)
txt = writer.extract_content(oeb_book, opts)
log.debug('\tReplacing newlines with selected type...')

4
src/calibre/ebooks/txt/txtml.py
View File

@ -35,6 +35,7 @@ BLOCK_STYLES = [
SPACE_TAGS = [
'td',
'br',
]
class TXTMLizer(object):
@ -178,8 +179,7 @@ class TXTMLizer(object):
text.append(u'\n\n')
if tag in SPACE_TAGS:
if not end.endswith('u ') and hasattr(elem, 'text') and elem.text:
text.append(u' ')
text.append(u' ')
# Process tags that contain text.
if hasattr(elem, 'text') and elem.text:

2
src/calibre/gui2/__init__.py
View File

@ -123,6 +123,8 @@ def _config():
help=_('Download social metadata (tags/rating/etc.)'))
c.add_opt('overwrite_author_title_metadata', default=True,
help=_('Overwrite author and title with new metadata'))
c.add_opt('auto_download_cover', default=False,
help=_('Automatically download the cover, if available'))
c.add_opt('enforce_cpu_limit', default=True,
help=_('Limit max simultaneous jobs to number of CPUs'))
c.add_opt('tag_browser_hidden_categories', default=set(),

4
src/calibre/gui2/actions/add.py
View File

@ -61,6 +61,7 @@ class AddAction(InterfaceAction):
self._adder = Adder(self.gui,
self.gui.library_view.model().db,
self.Dispatcher(self._files_added), spare_server=self.gui.spare_server)
self.gui.tags_view.disable_recounting = True
self._adder.add_recursive(root, single)
def add_recursive_single(self, *args):
@ -201,9 +202,11 @@ class AddAction(InterfaceAction):
self._adder = Adder(self.gui,
None if to_device else self.gui.library_view.model().db,
self.Dispatcher(self.__adder_func), spare_server=self.gui.spare_server)
self.gui.tags_view.disable_recounting = True
self._adder.add(paths)
def _files_added(self, paths=[], names=[], infos=[], on_card=None):
self.gui.tags_view.disable_recounting = False
if paths:
self.gui.upload_books(paths,
list(map(ascii_filename, names)),
@ -214,6 +217,7 @@ class AddAction(InterfaceAction):
self.gui.library_view.model().books_added(self._adder.number_of_books_added)
if hasattr(self.gui, 'db_images'):
self.gui.db_images.reset()
self.gui.tags_view.recount()
if getattr(self._adder, 'merged_books', False):
books = u'\n'.join([x if isinstance(x, unicode) else
x.decode(preferred_encoding, 'replace') for x in

65
src/calibre/gui2/actions/delete.py
View File

@ -5,13 +5,67 @@ __license__ = 'GPL v3'
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
from PyQt4.Qt import QMenu
from functools import partial
from PyQt4.Qt import QMenu, QObject, QTimer
from calibre.gui2 import error_dialog
from calibre.gui2.dialogs.delete_matching_from_device import DeleteMatchingFromDeviceDialog
from calibre.gui2.dialogs.confirm_delete import confirm
from calibre.gui2.actions import InterfaceAction
single_shot = partial(QTimer.singleShot, 10)
class MultiDeleter(QObject):
def __init__(self, gui, rows, callback):
from calibre.gui2.dialogs.progress import ProgressDialog
QObject.__init__(self, gui)
self.model = gui.library_view.model()
self.ids = list(map(self.model.id, rows))
self.gui = gui
self.failures = []
self.deleted_ids = []
self.callback = callback
single_shot(self.delete_one)
self.pd = ProgressDialog(_('Deleting...'), parent=gui,
cancelable=False, min=0, max=len(self.ids))
self.pd.setModal(True)
self.pd.show()
def delete_one(self):
if not self.ids:
self.cleanup()
return
id_ = self.ids.pop()
title = 'id:%d'%id_
try:
title_ = self.model.db.title(id_, index_is_id=True)
if title_:
title = title_
self.model.db.delete_book(id_, notify=False, commit=False)
self.deleted_ids.append(id_)
except:
import traceback
self.failures.append((id_, title, traceback.format_exc()))
single_shot(self.delete_one)
self.pd.value += 1
self.pd.set_msg(_('Deleted') + ' ' + title)
def cleanup(self):
self.pd.hide()
self.pd = None
self.model.db.commit()
self.model.db.clean()
self.model.books_deleted()
self.gui.tags_view.recount()
self.callback(self.deleted_ids)
if self.failures:
msg = ['==> '+x[1]+'\n'+x[2] for x in self.failures]
error_dialog(self.gui, _('Failed to delete'),
_('Failed to delete some books, click the Show Details button'
' for details.'), det_msg='\n\n'.join(msg), show=True)
class DeleteAction(InterfaceAction):
name = 'Remove Books'
@ -179,8 +233,13 @@ class DeleteAction(InterfaceAction):
row = None
if ci.isValid():
row = ci.row()
ids_deleted = view.model().delete_books(rows)
self.library_ids_deleted(ids_deleted, row)
if len(rows) < 5:
ids_deleted = view.model().delete_books(rows)
self.library_ids_deleted(ids_deleted, row)
else:
self.__md = MultiDeleter(self.gui, rows,
partial(self.library_ids_deleted, current_row=row))
else:
if not confirm('<p>'+_('The selected books will be '
'<b>permanently deleted</b> '

85
src/calibre/gui2/actions/edit_metadata.py
View File

@ -8,7 +8,7 @@ __docformat__ = 'restructuredtext en'
import os
from functools import partial
from PyQt4.Qt import Qt, QMenu
from PyQt4.Qt import Qt, QMenu, QModelIndex
from calibre.gui2 import error_dialog, config
from calibre.gui2.dialogs.metadata_single import MetadataSingleDialog
@ -16,6 +16,7 @@ from calibre.gui2.dialogs.metadata_bulk import MetadataBulkDialog
from calibre.gui2.dialogs.confirm_delete import confirm
from calibre.gui2.dialogs.tag_list_editor import TagListEditor
from calibre.gui2.actions import InterfaceAction
from calibre.utils.icu import sort_key
class EditMetadataAction(InterfaceAction):
@ -53,6 +54,10 @@ class EditMetadataAction(InterfaceAction):
mb.addAction(_('Merge into first selected book - keep others'),
partial(self.merge_books, safe_merge=True),
Qt.AltModifier+Qt.Key_M)
mb.addSeparator()
mb.addAction(_('Merge only formats into first selected book - delete others'),
partial(self.merge_books, merge_only_formats=True),
Qt.AltModifier+Qt.ShiftModifier+Qt.Key_M)
self.merge_menu = mb
self.action_merge.setMenu(mb)
md.addSeparator()
@ -126,20 +131,40 @@ class EditMetadataAction(InterfaceAction):
if bulk or (bulk is None and len(rows) > 1):
return self.edit_bulk_metadata(checked)
def accepted(id):
self.gui.library_view.model().refresh_ids([id])
row_list = [r.row() for r in rows]
current_row = 0
changed = set([])
db = self.gui.library_view.model().db
for row in rows:
self.gui.iactions['View'].metadata_view_id = self.gui.library_view.model().db.id(row.row())
d = MetadataSingleDialog(self.gui, row.row(),
self.gui.library_view.model().db,
accepted_callback=accepted,
cancel_all=rows.index(row) < len(rows)-1)
d.view_format.connect(self.gui.iactions['View'].metadata_view_format)
d.exec_()
if d.cancel_all:
if len(row_list) == 1:
cr = row_list[0]
row_list = \
list(range(self.gui.library_view.model().rowCount(QModelIndex())))
current_row = row_list.index(cr)
while True:
prev = next_ = None
if current_row > 0:
prev = db.title(row_list[current_row-1])
if current_row < len(row_list) - 1:
next_ = db.title(row_list[current_row+1])
d = MetadataSingleDialog(self.gui, row_list[current_row], db,
prev=prev, next_=next_)
d.view_format.connect(lambda
fmt:self.gui.iactions['View'].view_format(row_list[current_row],
fmt))
if d.exec_() != d.Accepted:
d.view_format.disconnect()
break
if rows:
d.view_format.disconnect()
changed.add(d.id)
if d.row_delta == 0:
break
current_row += d.row_delta
if changed:
self.gui.library_view.model().refresh_ids(list(changed))
current = self.gui.library_view.currentIndex()
m = self.gui.library_view.model()
if self.gui.cover_flow:
@ -185,7 +210,7 @@ class EditMetadataAction(InterfaceAction):
self.gui.library_view.select_rows(ids)
# Merge books {{{
def merge_books(self, safe_merge=False):
def merge_books(self, safe_merge=False, merge_only_formats=False):
'''
Merge selected books in library.
'''
@ -199,6 +224,12 @@ class EditMetadataAction(InterfaceAction):
return error_dialog(self.gui, _('Cannot merge books'),
_('At least two books must be selected for merging'),
show=True)
if len(rows) > 5:
if not confirm('<p>'+_('You are about to merge more than 5 books. '
'Are you <b>sure</b> you want to proceed?')
+'</p>', 'merge_too_many_books', self.gui):
return
dest_id, src_books, src_ids = self.books_to_merge(rows)
title = self.gui.library_view.model().db.title(dest_id, index_is_id=True)
if safe_merge:
@ -213,6 +244,22 @@ class EditMetadataAction(InterfaceAction):
return
self.add_formats(dest_id, src_books)
self.merge_metadata(dest_id, src_ids)
elif merge_only_formats:
if not confirm('<p>'+_(
'Book formats from the selected books will be merged '
'into the <b>first selected book</b> (%s). '
'Metadata in the first selected book will not be changed.'
'Author, Title, ISBN and all other metadata will <i>not</i> be merged.<br><br>'
'After merger the second and subsequently '
'selected books, with any metadata they have will be <b>deleted</b>. <br><br>'
'All book formats of the first selected book will be kept '
'and any duplicate formats in the second and subsequently selected books '
'will be permanently <b>deleted</b> from your calibre library.<br><br> '
'Are you <b>sure</b> you want to proceed?')%title
+'</p>', 'merge_only_formats', self.gui):
return
self.add_formats(dest_id, src_books)
self.delete_books_after_merge(src_ids)
else:
if not confirm('<p>'+_(
'Book formats and metadata from the selected books will be merged '
@ -222,15 +269,10 @@ class EditMetadataAction(InterfaceAction):
'subsequently selected books will be <b>deleted</b>. <br><br>'
'All book formats of the first selected book will be kept '
'and any duplicate formats in the second and subsequently selected books '
'will be permanently <b>deleted</b> from your computer.<br><br> '
'will be permanently <b>deleted</b> from your calibre library.<br><br> '
'Are you <b>sure</b> you want to proceed?')%title
+'</p>', 'merge_books', self.gui):
return
if len(rows)>5:
if not confirm('<p>'+_('You are about to merge more than 5 books. '
'Are you <b>sure</b> you want to proceed?')
+'</p>', 'merge_too_many_books', self.gui):
return
self.add_formats(dest_id, src_books)
self.merge_metadata(dest_id, src_ids)
self.delete_books_after_merge(src_ids)
@ -348,8 +390,7 @@ class EditMetadataAction(InterfaceAction):
def edit_device_collections(self, view, oncard=None):
model = view.model()
result = model.get_collections_with_ids()
compare = (lambda x,y:cmp(x.lower(), y.lower()))
d = TagListEditor(self.gui, tag_to_match=None, data=result, compare=compare)
d = TagListEditor(self.gui, tag_to_match=None, data=result, key=sort_key)
d.exec_()
if d.result() == d.Accepted:
to_rename = d.to_rename # dict of new text to old ids

3
src/calibre/gui2/actions/show_book_details.py
View File

@ -29,5 +29,6 @@ class ShowBookDetailsAction(InterfaceAction):
return
index = self.gui.library_view.currentIndex()
if index.isValid():
BookInfo(self.gui, self.gui.library_view, index).show()
BookInfo(self.gui, self.gui.library_view, index,
self.gui.iactions['View'].view_format_by_id).show()

9
src/calibre/gui2/actions/view.py
View File

@ -26,7 +26,6 @@ class ViewAction(InterfaceAction):
def genesis(self):
self.persistent_files = []
self.metadata_view_id = None
self.qaction.triggered.connect(self.view_book)
self.view_menu = QMenu()
self.view_menu.addAction(_('View'), partial(self.view_book, False))
@ -51,14 +50,6 @@ class ViewAction(InterfaceAction):
if fmt_path:
self._view_file(fmt_path)
def metadata_view_format(self, fmt):
fmt_path = self.gui.library_view.model().db.\
format_abspath(self.metadata_view_id,
fmt, index_is_id=True)
if fmt_path:
self._view_file(fmt_path)
def book_downloaded_for_viewing(self, job):
if job.failed:
self.gui.device_job_exception(job)

181
src/calibre/gui2/add.py
View File

@ -3,41 +3,55 @@ UI for adding books to the database and saving books to disk
'''
import os, shutil, time
from Queue import Queue, Empty
from threading import Thread
from functools import partial
from PyQt4.Qt import QThread, SIGNAL, QObject, QTimer, Qt, \
QProgressDialog
from PyQt4.Qt import QThread, QObject, Qt, QProgressDialog, pyqtSignal, QTimer
from calibre.gui2.dialogs.progress import ProgressDialog
from calibre.gui2 import question_dialog, error_dialog, info_dialog
from calibre.ebooks.metadata.opf2 import OPF
from calibre.ebooks.metadata import MetaInformation
from calibre.constants import preferred_encoding, filesystem_encoding
from calibre.constants import preferred_encoding, filesystem_encoding, DEBUG
from calibre.utils.config import prefs
from calibre import prints
single_shot = partial(QTimer.singleShot, 75)
class DuplicatesAdder(QObject): # {{{
added = pyqtSignal(object)
adding_done = pyqtSignal()
class DuplicatesAdder(QThread): # {{{
# Add duplicate books
def __init__(self, parent, db, duplicates, db_adder):
QThread.__init__(self, parent)
QObject.__init__(self, parent)
self.db, self.db_adder = db, db_adder
self.duplicates = duplicates
self.duplicates = list(duplicates)
self.count = 0
single_shot(self.add_one)
def add_one(self):
if not self.duplicates:
self.adding_done.emit()
return
mi, cover, formats = self.duplicates.pop()
formats = [f for f in formats if not f.lower().endswith('.opf')]
id = self.db.create_book_entry(mi, cover=cover,
add_duplicates=True)
# here we add all the formats for dupe book record created above
self.db_adder.add_formats(id, formats)
self.db_adder.number_of_books_added += 1
self.count += 1
self.added.emit(self.count)
single_shot(self.add_one)
def run(self):
count = 1
for mi, cover, formats in self.duplicates:
formats = [f for f in formats if not f.lower().endswith('.opf')]
id = self.db.create_book_entry(mi, cover=cover,
add_duplicates=True)
# here we add all the formats for dupe book record created above
self.db_adder.add_formats(id, formats)
self.db_adder.number_of_books_added += 1
self.emit(SIGNAL('added(PyQt_PyObject)'), count)
count += 1
self.emit(SIGNAL('adding_done()'))
# }}}
class RecursiveFind(QThread): # {{{
update = pyqtSignal(object)
found = pyqtSignal(object)
def __init__(self, parent, db, root, single):
QThread.__init__(self, parent)
self.db = db
@ -50,8 +64,8 @@ class RecursiveFind(QThread): # {{{
for dirpath in os.walk(root):
if self.canceled:
return
self.emit(SIGNAL('update(PyQt_PyObject)'),
_('Searching in')+' '+dirpath[0])
self.update.emit(
_('Searching in')+' '+dirpath[0])
self.books += list(self.db.find_books_in_directory(dirpath[0],
self.single_book_per_directory))
@ -71,46 +85,55 @@ class RecursiveFind(QThread): # {{{
msg = unicode(err)
except:
msg = repr(err)
self.emit(SIGNAL('found(PyQt_PyObject)'), msg)
self.found.emit(msg)
return
self.books = [formats for formats in self.books if formats]
if not self.canceled:
self.emit(SIGNAL('found(PyQt_PyObject)'), self.books)
self.found.emit(self.books)
# }}}
class DBAdder(Thread): # {{{
class DBAdder(QObject): # {{{
def __init__(self, parent, db, ids, nmap):
QObject.__init__(self, parent)
def __init__(self, db, ids, nmap):
self.db, self.ids, self.nmap = db, dict(**ids), dict(**nmap)
self.end = False
self.critical = {}
self.number_of_books_added = 0
self.duplicates = []
self.names, self.paths, self.infos = [], [], []
Thread.__init__(self)
self.daemon = True
self.input_queue = Queue()
self.output_queue = Queue()
self.merged_books = set([])
def run(self):
while not self.end:
try:
id, opf, cover = self.input_queue.get(True, 0.2)
except Empty:
continue
name = self.nmap.pop(id)
title = None
try:
title = self.add(id, opf, cover, name)
except:
import traceback
self.critical[name] = traceback.format_exc()
title = name
self.output_queue.put(title)
def end(self):
self.input_queue.put((None, None, None))
def start(self):
try:
id, opf, cover = self.input_queue.get_nowait()
except Empty:
single_shot(self.start)
return
if id is None and opf is None and cover is None:
return
name = self.nmap.pop(id)
title = None
if DEBUG:
st = time.time()
try:
title = self.add(id, opf, cover, name)
except:
import traceback
self.critical[name] = traceback.format_exc()
title = name
self.output_queue.put(title)
if DEBUG:
prints('Added', title, 'to db in:', time.time() - st, 'seconds')
single_shot(self.start)
def process_formats(self, opf, formats):
imp = opf[:-4]+'.import'
@ -201,10 +224,10 @@ class Adder(QObject): # {{{
self.pd.setModal(True)
self.pd.show()
self._parent = parent
self.rfind = self.worker = self.timer = None
self.rfind = self.worker = None
self.callback = callback
self.callback_called = False
self.connect(self.pd, SIGNAL('canceled()'), self.canceled)
self.pd.canceled_signal.connect(self.canceled)
def add_recursive(self, root, single=True):
self.path = root
@ -213,10 +236,8 @@ class Adder(QObject): # {{{
self.pd.set_max(0)
self.pd.value = 0
self.rfind = RecursiveFind(self, self.db, root, single)
self.connect(self.rfind, SIGNAL('update(PyQt_PyObject)'),
self.pd.set_msg, Qt.QueuedConnection)
self.connect(self.rfind, SIGNAL('found(PyQt_PyObject)'),
self.add, Qt.QueuedConnection)
self.rfind.update.connect(self.pd.set_msg, type=Qt.QueuedConnection)
self.rfind.found.connect(self.add, type=Qt.QueuedConnection)
self.rfind.start()
def add(self, books):
@ -246,12 +267,12 @@ class Adder(QObject): # {{{
self.pd.set_min(0)
self.pd.set_max(len(self.ids))
self.pd.value = 0
self.db_adder = DBAdder(self.db, self.ids, self.nmap)
self.db_adder = DBAdder(self, self.db, self.ids, self.nmap)
self.db_adder.start()
self.last_added_at = time.time()
self.entry_count = len(self.ids)
self.continue_updating = True
QTimer.singleShot(200, self.update)
single_shot(self.update)
def canceled(self):
self.continue_updating = False
@ -260,14 +281,14 @@ class Adder(QObject): # {{{
if self.worker is not None:
self.worker.canceled = True
if hasattr(self, 'db_adder'):
self.db_adder.end = True
self.db_adder.end()
self.pd.hide()
if not self.callback_called:
self.callback(self.paths, self.names, self.infos)
self.callback_called = True
def duplicates_processed(self):
self.db_adder.end = True
self.db_adder.end()
if not self.callback_called:
self.callback(self.paths, self.names, self.infos)
self.callback_called = True
@ -300,7 +321,7 @@ class Adder(QObject): # {{{
if (time.time() - self.last_added_at) > self.ADD_TIMEOUT:
self.continue_updating = False
self.pd.hide()
self.db_adder.end = True
self.db_adder.end()
if not self.callback_called:
self.callback([], [], [])
self.callback_called = True
@ -311,7 +332,7 @@ class Adder(QObject): # {{{
'find the problem book.'), show=True)
if self.continue_updating:
QTimer.singleShot(200, self.update)
single_shot(self.update)
def process_duplicates(self):
@ -332,11 +353,8 @@ class Adder(QObject): # {{{
self.__p_d = pd
self.__d_a = DuplicatesAdder(self._parent, self.db, duplicates,
self.db_adder)
self.connect(self.__d_a, SIGNAL('added(PyQt_PyObject)'),
pd.setValue)
self.connect(self.__d_a, SIGNAL('adding_done()'),
self.duplicates_processed)
self.__d_a.start()
self.__d_a.added.connect(pd.setValue)
self.__d_a.adding_done.connect(self.duplicates_processed)
else:
return self.duplicates_processed()
@ -407,14 +425,12 @@ class Saver(QObject): # {{{
self.worker = SaveWorker(self.rq, db, self.ids, path, self.opts,
spare_server=self.spare_server)
self.pd.canceled_signal.connect(self.canceled)
self.timer = QTimer(self)
self.connect(self.timer, SIGNAL('timeout()'), self.update)
self.timer.start(200)
self.continue_updating = True
single_shot(self.update)
def canceled(self):
if self.timer is not None:
self.timer.stop()
self.continue_updating = False
if self.worker is not None:
self.worker.canceled = True
self.pd.hide()
@ -424,14 +440,38 @@ class Saver(QObject): # {{{
def update(self):
if not self.ids or not self.worker.is_alive():
self.timer.stop()
if not self.continue_updating:
return
if not self.worker.is_alive():
# Check that all ids were processed
while self.ids:
# Get all queued results since worker is dead
before = len(self.ids)
self.get_result()
if before == len(self.ids):
# No results available => worker died unexpectedly
for i in list(self.ids):
self.failures.add(('id:%d'%i, 'Unknown error'))
self.ids.remove(i)
if not self.ids:
self.continue_updating = False
self.pd.hide()
if not self.callback_called:
self.callback(self.worker.path, self.failures, self.worker.error)
try:
# Give the worker time to clean up and set worker.error
self.worker.join(2)
except:
pass # The worker was not yet started
self.callback_called = True
return
self.callback(self.worker.path, self.failures, self.worker.error)
if self.continue_updating:
self.get_result()
single_shot(self.update)
def get_result(self):
try:
id, title, ok, tb = self.rq.get_nowait()
except Empty:
@ -441,6 +481,7 @@ class Saver(QObject): # {{{
if not isinstance(title, unicode):
title = str(title).decode(preferred_encoding, 'replace')
self.pd.set_msg(_('Saved')+' '+title)
if not ok:
self.failures.add((title, tb))
# }}}

5
src/calibre/gui2/book_details.py
View File

@ -19,6 +19,7 @@ from calibre.ebooks import BOOK_EXTENSIONS
from calibre.constants import preferred_encoding
from calibre.library.comments import comments_to_html
from calibre.gui2 import config, open_local_file
from calibre.utils.icu import sort_key
# render_rows(data) {{{
WEIGHTS = collections.defaultdict(lambda : 100)
@ -31,8 +32,8 @@ WEIGHTS[_('Tags')] = 4
def render_rows(data):
keys = data.keys()
# First sort by name. The WEIGHTS sort will preserve this sub-order
keys.sort(cmp=lambda x, y: cmp(x.lower(), y.lower()))
keys.sort(cmp=lambda x, y: cmp(WEIGHTS[x], WEIGHTS[y]))
keys.sort(key=sort_key)
keys.sort(key=lambda x: WEIGHTS[x])
rows = []
for key in keys:
txt = data[key]

4
src/calibre/gui2/convert/fb2_output.py
View File

@ -17,8 +17,6 @@ class PluginWidget(Widget, Ui_Form):
ICON = I('mimetypes/fb2.png')
def __init__(self, parent, get_option, get_help, db=None, book_id=None):
Widget.__init__(self, parent, ['inline_toc', 'sectionize_chapters',
'sectionize_chapters_using_file_structure', 'h1_to_title',
'h2_to_title', 'h3_to_title'])
Widget.__init__(self, parent, ['h1_to_title', 'h2_to_title', 'h3_to_title'])
self.db, self.book_id = db, book_id
self.initialize_options(get_option, get_help, db, book_id)

27
src/calibre/gui2/convert/fb2_output.ui
View File

@ -14,7 +14,7 @@
<string>Form</string>
</property>
<layout class="QGridLayout" name="gridLayout">
<item row="6" column="0">
<item row="3" column="0">
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
@ -28,41 +28,20 @@
</spacer>
</item>
<item row="0" column="0">
<widget class="QCheckBox" name="opt_inline_toc">
<property name="text">
<string>&amp;Inline TOC</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QCheckBox" name="opt_sectionize_chapters">
<property name="text">
<string>Sectionize Chapters (Use with care!)</string>
</property>
</widget>
</item>
<item row="2" column="0">
<widget class="QCheckBox" name="opt_sectionize_chapters_using_file_structure">
<property name="text">
<string>Sectionize Chapters using file structure</string>
</property>
</widget>
</item>
<item row="3" column="0">
<widget class="QCheckBox" name="opt_h1_to_title">
<property name="text">
<string>Wrap h1 tags with &lt;title&gt; elements</string>
</property>
</widget>
</item>
<item row="4" column="0">
<item row="1" column="0">
<widget class="QCheckBox" name="opt_h2_to_title">
<property name="text">
<string>Wrap h2 tags with &lt;title&gt; elements</string>
</property>
</widget>
</item>
<item row="5" column="0">
<item row="2" column="0">
<widget class="QCheckBox" name="opt_h3_to_title">
<property name="text">
<string>Wrap h3 tags with &lt;title&gt; elements</string>

7
src/calibre/gui2/convert/metadata.py
View File

@ -17,6 +17,7 @@ from calibre.ebooks.metadata import authors_to_string, string_to_authors, \
from calibre.ebooks.metadata.opf2 import metadata_to_opf
from calibre.ptempfile import PersistentTemporaryFile
from calibre.gui2.convert import Widget
from calibre.utils.icu import sort_key
def create_opf_file(db, book_id):
mi = db.get_metadata(book_id, index_is_id=True)
@ -102,7 +103,7 @@ class MetadataWidget(Widget, Ui_Form):
def initalize_authors(self):
all_authors = self.db.all_authors()
all_authors.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_authors.sort(key=lambda x : sort_key(x[1]))
for i in all_authors:
id, name = i
@ -117,7 +118,7 @@ class MetadataWidget(Widget, Ui_Form):
def initialize_series(self):
all_series = self.db.all_series()
all_series.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_series.sort(key=lambda x : sort_key(x[1]))
for i in all_series:
id, name = i
@ -126,7 +127,7 @@ class MetadataWidget(Widget, Ui_Form):
def initialize_publisher(self):
all_publishers = self.db.all_publishers()
all_publishers.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_publishers.sort(key=lambda x : sort_key(x[1]))
for i in all_publishers:
id, name = i

22
src/calibre/gui2/convert/pml_output.py Normal file
View File

@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
__license__ = 'GPL 3'
__copyright__ = '2009, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
from calibre.gui2.convert.pmlz_output_ui import Ui_Form
from calibre.gui2.convert import Widget
format_model = None
class PluginWidget(Widget, Ui_Form):
TITLE = _('PMLZ Output')
HELP = _('Options specific to')+' PMLZ '+_('output')
COMMIT_NAME = 'pmlz_output'
ICON = I('mimetypes/unknown.png')
def __init__(self, parent, get_option, get_help, db=None, book_id=None):
Widget.__init__(self, parent, ['inline_toc', 'full_image_depth'])
self.db, self.book_id = db, book_id
self.initialize_options(get_option, get_help, db, book_id)

48
src/calibre/gui2/convert/pmlz_output.ui Normal file
View File

@ -0,0 +1,48 @@
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>Form</class>
<widget class="QWidget" name="Form">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>400</width>
<height>300</height>
</rect>
</property>
<property name="windowTitle">
<string>Form</string>
</property>
<layout class="QGridLayout" name="gridLayout">
<item row="2" column="0">
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>246</height>
</size>
</property>
</spacer>
</item>
<item row="0" column="0">
<widget class="QCheckBox" name="opt_inline_toc">
<property name="text">
<string>&amp;Inline TOC</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QCheckBox" name="opt_full_image_depth">
<property name="text">
<string>Do not reduce image size and depth</string>
</property>
</widget>
</item>
</layout>
</widget>
<resources/>
<connections/>
</ui>

2
src/calibre/gui2/convert/txt_output.py
View File

@ -21,7 +21,7 @@ class PluginWidget(Widget, Ui_Form):
def __init__(self, parent, get_option, get_help, db=None, book_id=None):
Widget.__init__(self, parent,
['newline', 'max_line_length', 'force_max_line_length',
'inline_toc'])
'inline_toc', 'markdown_format', 'keep_links', 'keep_image_references'])
self.db, self.book_id = db, book_id
self.initialize_options(get_option, get_help, db, book_id)

25
src/calibre/gui2/convert/txt_output.ui
View File

@ -6,7 +6,7 @@
<rect>
<x>0</x>
<y>0</y>
<width>400</width>
<width>477</width>
<height>300</height>
</rect>
</property>
@ -27,7 +27,7 @@
<item row="0" column="1">
<widget class="QComboBox" name="opt_newline"/>
</item>
<item row="4" column="0">
<item row="7" column="0">
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
@ -67,6 +67,27 @@
</property>
</widget>
</item>
<item row="4" column="0">
<widget class="QCheckBox" name="opt_markdown_format">
<property name="text">
<string>Apply Markdown formatting to text</string>
</property>
</widget>
</item>
<item row="5" column="0">
<widget class="QCheckBox" name="opt_keep_links">
<property name="text">
<string>Do not remove links (&lt;a&gt; tags) before processing</string>
</property>
</widget>
</item>
<item row="6" column="0">
<widget class="QCheckBox" name="opt_keep_image_references">
<property name="text">
<string>Do not remove image references before processing</string>
</property>
</widget>
</item>
</layout>
</widget>
<resources/>

120
src/calibre/gui2/custom_column_widgets.py
View File

@ -15,8 +15,9 @@ from PyQt4.Qt import QComboBox, QLabel, QSpinBox, QDoubleSpinBox, QDateEdit, \
from calibre.utils.date import qt_to_dt, now
from calibre.gui2.widgets import TagsLineEdit, EnComboBox
from calibre.gui2 import UNDEFINED_QDATE
from calibre.gui2 import UNDEFINED_QDATE, error_dialog
from calibre.utils.config import tweaks
from calibre.utils.icu import sort_key
class Base(object):
@ -207,7 +208,7 @@ class Text(Base):
def setup_ui(self, parent):
values = self.all_values = list(self.db.all_custom(num=self.col_id))
values.sort(cmp = lambda x,y: cmp(x.lower(), y.lower()))
values.sort(key=sort_key)
if self.col_metadata['is_multiple']:
w = TagsLineEdit(parent, values)
w.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)
@ -256,7 +257,7 @@ class Series(Base):
def setup_ui(self, parent):
values = self.all_values = list(self.db.all_custom(num=self.col_id))
values.sort(cmp = lambda x,y: cmp(x.lower(), y.lower()))
values.sort(key=sort_key)
w = EnComboBox(parent)
w.setSizeAdjustPolicy(w.AdjustToMinimumContentsLengthWithIcon)
w.setMinimumContentsLength(25)
@ -310,6 +311,49 @@ class Series(Base):
self.db.set_custom(book_id, val, extra=s_index,
num=self.col_id, notify=notify, commit=False)
class Enumeration(Base):
def setup_ui(self, parent):
self.parent = parent
self.widgets = [QLabel('&'+self.col_metadata['name']+':', parent),
QComboBox(parent)]
w = self.widgets[1]
vals = self.col_metadata['display']['enum_values']
w.addItem('')
for v in vals:
w.addItem(v)
def initialize(self, book_id):
val = self.db.get_custom(book_id, num=self.col_id, index_is_id=True)
val = self.normalize_db_val(val)
self.initial_val = val
idx = self.widgets[1].findText(val)
if idx < 0:
error_dialog(self.parent, '',
_('The enumeration "{0}" contains an invalid value '
'that will be set to the default').format(
self.col_metadata['name']),
show=True, show_copy_button=False)
idx = 0
self.widgets[1].setCurrentIndex(idx)
def setter(self, val):
self.widgets[1].setCurrentIndex(self.widgets[1].findText(val))
def getter(self):
return unicode(self.widgets[1].currentText())
def normalize_db_val(self, val):
if val is None:
val = ''
return val
def normalize_ui_val(self, val):
if not val:
val = None
return val
widgets = {
'bool' : Bool,
'rating' : Rating,
@ -319,13 +363,13 @@ widgets = {
'text' : Text,
'comments': Comments,
'series': Series,
'enumeration': Enumeration
}
def field_sort(y, z, x=None):
m1, m2 = x[y], x[z]
def field_sort_key(y, x=None):
m1 = x[y]
n1 = 'zzzzz' if m1['datatype'] == 'comments' else m1['name']
n2 = 'zzzzz' if m2['datatype'] == 'comments' else m2['name']
return cmp(n1.lower(), n2.lower())
return sort_key(n1)
def populate_metadata_page(layout, db, book_id, bulk=False, two_column=False, parent=None):
def widget_factory(type, col):
@ -337,7 +381,7 @@ def populate_metadata_page(layout, db, book_id, bulk=False, two_column=False, pa
return w
x = db.custom_column_num_map
cols = list(x)
cols.sort(cmp=partial(field_sort, x=x))
cols.sort(key=partial(field_sort_key, x=x))
count_non_comment = len([c for c in cols if x[c]['datatype'] != 'comments'])
layout.setColumnStretch(1, 10)
@ -482,7 +526,7 @@ class BulkSeries(BulkBase):
def setup_ui(self, parent):
values = self.all_values = list(self.db.all_custom(num=self.col_id))
values.sort(cmp = lambda x,y: cmp(x.lower(), y.lower()))
values.sort(key=sort_key)
w = EnComboBox(parent)
w.setSizeAdjustPolicy(w.AdjustToMinimumContentsLengthWithIcon)
w.setMinimumContentsLength(25)
@ -551,6 +595,61 @@ class BulkSeries(BulkBase):
self.db.set_custom_bulk(book_ids, val, extras=extras,
num=self.col_id, notify=notify)
class BulkEnumeration(BulkBase, Enumeration):
def get_initial_value(self, book_ids):
value = None
ret_value = None
dialog_shown = False
for book_id in book_ids:
val = self.db.get_custom(book_id, num=self.col_id, index_is_id=True)
if val and val not in self.col_metadata['display']['enum_values']:
if not dialog_shown:
error_dialog(self.parent, '',
_('The enumeration "{0}" contains invalid values '
'that will not appear in the list').format(
self.col_metadata['name']),
show=True, show_copy_button=False)
dialog_shown = True
ret_value = ' nochange '
elif value is not None and value != val:
ret_value = ' nochange '
value = val
if ret_value is None:
return value
return ret_value
def setup_ui(self, parent):
self.parent = parent
self.widgets = [QLabel('&'+self.col_metadata['name']+':', parent),
QComboBox(parent)]
w = self.widgets[1]
vals = self.col_metadata['display']['enum_values']
w.addItem('Do Not Change')
w.addItem('')
for v in vals:
w.addItem(v)
def getter(self):
if self.widgets[1].currentIndex() == 0:
return ' nochange '
return unicode(self.widgets[1].currentText())
def setter(self, val):
if val == ' nochange ':
self.widgets[1].setCurrentIndex(0)
else:
if val is None:
self.widgets[1].setCurrentIndex(1)
else:
self.widgets[1].setCurrentIndex(self.widgets[1].findText(val))
def commit(self, book_ids, notify=False):
val = self.gui_val
val = self.normalize_ui_val(val)
if val != self.initial_val and val != ' nochange ':
self.db.set_custom_bulk(book_ids, val, num=self.col_id, notify=notify)
class RemoveTags(QWidget):
def __init__(self, parent, values):
@ -579,7 +678,7 @@ class BulkText(BulkBase):
def setup_ui(self, parent):
values = self.all_values = list(self.db.all_custom(num=self.col_id))
values.sort(cmp = lambda x,y: cmp(x.lower(), y.lower()))
values.sort(key=sort_key)
if self.col_metadata['is_multiple']:
w = TagsLineEdit(parent, values)
w.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)
@ -656,4 +755,5 @@ bulk_widgets = {
'datetime': BulkDateTime,
'text' : BulkText,
'series': BulkSeries,
'enumeration': BulkEnumeration,
}

8
src/calibre/gui2/dialogs/book_info.py
View File

@ -15,12 +15,13 @@ from calibre.library.comments import comments_to_html
class BookInfo(QDialog, Ui_BookInfo):
def __init__(self, parent, view, row):
def __init__(self, parent, view, row, view_func):
QDialog.__init__(self, parent)
Ui_BookInfo.__init__(self)
self.setupUi(self)
self.cover_pixmap = None
self.comments.sizeHint = self.comments_size_hint
self.view_func = view_func
desktop = QCoreApplication.instance().desktop()
screen_height = desktop.availableGeometry().height() - 100
@ -58,10 +59,7 @@ class BookInfo(QDialog, Ui_BookInfo):
if os.sep in path:
open_local_file(path)
else:
path = self.view.model().db.format_abspath(self.current_row, path)
if path is not None:
open_local_file(path)
self.view_func(self.view.model().id(self.current_row), path)
def next(self):
row = self.view.currentIndex().row()

5
src/calibre/gui2/dialogs/comments_dialog.py
View File

@ -3,7 +3,7 @@ __copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'
__license__ = 'GPL v3'
from PyQt4.Qt import Qt, QDialog
from PyQt4.Qt import Qt, QDialog, QDialogButtonBox
from calibre.gui2.dialogs.comments_dialog_ui import Ui_CommentsDialog
class CommentsDialog(QDialog, Ui_CommentsDialog):
@ -20,3 +20,6 @@ class CommentsDialog(QDialog, Ui_CommentsDialog):
if text is not None:
self.textbox.setPlainText(text)
self.textbox.setTabChangesFocus(True)
self.buttonBox.button(QDialogButtonBox.Ok).setText(_('&OK'))
self.buttonBox.button(QDialogButtonBox.Cancel).setText(_('&Cancel'))

10
src/calibre/gui2/dialogs/comments_dialog.ui
View File

@ -19,15 +19,6 @@
<property name="windowTitle">
<string>Edit Comments</string>
</property>
<widget class="QWidget" name="verticalLayoutWidget">
<property name="geometry">
<rect>
<x>10</x>
<y>10</y>
<width>311</width>
<height>211</height>
</rect>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QPlainTextEdit" name="textbox"/>
@ -43,7 +34,6 @@
</widget>
</item>
</layout>
</widget>
</widget>
<resources/>
<connections>

1
src/calibre/gui2/dialogs/confirm_delete.py
View File

@ -31,4 +31,5 @@ def confirm(msg, name, parent=None, pixmap='dialog_warning.png'):
d = Dialog(msg, name, parent)
d.label.setPixmap(QPixmap(I(pixmap)))
d.setWindowIcon(QIcon(I(pixmap)))
d.resize(d.sizeHint())
return d.exec_() == d.Accepted

6
src/calibre/gui2/dialogs/edit_authors_dialog.py
View File

@ -8,13 +8,14 @@ from PyQt4.Qt import Qt, QDialog, QTableWidgetItem, QAbstractItemView
from calibre.ebooks.metadata import author_to_author_sort
from calibre.gui2 import error_dialog
from calibre.gui2.dialogs.edit_authors_dialog_ui import Ui_EditAuthorsDialog
from calibre.utils.icu import sort_key, strcmp
class tableItem(QTableWidgetItem):
def __ge__(self, other):
return unicode(self.text()).lower() >= unicode(other.text()).lower()
return sort_key(unicode(self.text())) >= sort_key(unicode(other.text()))
def __lt__(self, other):
return unicode(self.text()).lower() < unicode(other.text()).lower()
return sort_key(unicode(self.text())) < sort_key(unicode(other.text()))
class EditAuthorsDialog(QDialog, Ui_EditAuthorsDialog):
@ -36,6 +37,7 @@ class EditAuthorsDialog(QDialog, Ui_EditAuthorsDialog):
self.authors = {}
auts = db.get_authors_with_ids()
self.table.setRowCount(len(auts))
setattr(self.table, '__lt__', lambda x, y: True if strcmp(x, y) < 0 else False)
select_item = None
for row, (id, author, sort) in enumerate(auts):
author = author.replace('|', ',')

22
src/calibre/gui2/dialogs/fetch_metadata.py
View File

@ -9,7 +9,7 @@ from threading import Thread
from PyQt4.QtCore import Qt, QObject, SIGNAL, QVariant, pyqtSignal, \
QAbstractTableModel, QCoreApplication, QTimer
from PyQt4.QtGui import QDialog, QItemSelectionModel
from PyQt4.QtGui import QDialog, QItemSelectionModel, QIcon
from calibre.gui2.dialogs.fetch_metadata_ui import Ui_FetchMetadata
from calibre.gui2 import error_dialog, NONE, info_dialog, config
@ -42,13 +42,14 @@ class Matches(QAbstractTableModel):
def __init__(self, matches):
self.matches = matches
self.yes_icon = QVariant(QIcon(I('ok.png')))
QAbstractTableModel.__init__(self)
def rowCount(self, *args):
return len(self.matches)
def columnCount(self, *args):
return 6
return 8
def headerData(self, section, orientation, role):
if role != Qt.DisplayRole:
@ -61,6 +62,8 @@ class Matches(QAbstractTableModel):
elif section == 3: text = _("Publisher")
elif section == 4: text = _("ISBN")
elif section == 5: text = _("Published")
elif section == 6: text = _("Has Cover")
elif section == 7: text = _("Has Summary")
return QVariant(text)
else:
@ -71,8 +74,8 @@ class Matches(QAbstractTableModel):
def data(self, index, role):
row, col = index.row(), index.column()
book = self.matches[row]
if role == Qt.DisplayRole:
book = self.matches[row]
res = None
if col == 0:
res = book.title
@ -90,6 +93,11 @@ class Matches(QAbstractTableModel):
if not res:
return NONE
return QVariant(res)
elif role == Qt.DecorationRole:
if col == 6 and book.has_cover:
return self.yes_icon
if col == 7 and book.comments:
return self.yes_icon
return NONE
class FetchMetadata(QDialog, Ui_FetchMetadata):
@ -131,7 +139,7 @@ class FetchMetadata(QDialog, Ui_FetchMetadata):
self.fetch_metadata()
self.opt_get_social_metadata.setChecked(config['get_social_metadata'])
self.opt_overwrite_author_title_metadata.setChecked(config['overwrite_author_title_metadata'])
self.opt_auto_download_cover.setChecked(config['auto_download_cover'])
def show_summary(self, current, *args):
row = current.row()
@ -213,6 +221,12 @@ class FetchMetadata(QDialog, Ui_FetchMetadata):
_hung_fetchers.add(self.fetcher)
if hasattr(self, '_hangcheck') and self._hangcheck.isActive():
self._hangcheck.stop()
# Save value of auto_download_cover, since this is the only place it can
# be set. The values of the other options can be set in
# Preferences->Behavior and should not be set here as they affect bulk
# downloading as well.
if self.opt_auto_download_cover.isChecked() != config['auto_download_cover']:
config.set('auto_download_cover', self.opt_auto_download_cover.isChecked())
def __enter__(self, *args):
return self

351
src/calibre/gui2/dialogs/fetch_metadata.ui
View File

@ -1,172 +1,179 @@
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>FetchMetadata</class>
<widget class="QDialog" name="FetchMetadata">
<property name="windowModality">
<enum>Qt::WindowModal</enum>
</property>
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>830</width>
<height>642</height>
</rect>
</property>
<property name="windowTitle">
<string>Fetch metadata</string>
</property>
<property name="windowIcon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/metadata.png</normaloff>:/images/metadata.png</iconset>
</property>
<layout class="QVBoxLayout">
<item>
<widget class="QLabel" name="tlabel">
<property name="text">
<string>&lt;p&gt;calibre can find metadata for your books from two locations: &lt;b&gt;Google Books&lt;/b&gt; and &lt;b&gt;isbndb.com&lt;/b&gt;. &lt;p&gt;To use isbndb.com you must sign up for a &lt;a href=&quot;http://www.isbndb.com&quot;&gt;free account&lt;/a&gt; and enter your access key below.</string>
</property>
<property name="alignment">
<set>Qt::AlignCenter</set>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
<property name="openExternalLinks">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<layout class="QHBoxLayout">
<item>
<widget class="QLabel" name="label_2">
<property name="text">
<string>&amp;Access Key:</string>
</property>
<property name="buddy">
<cstring>key</cstring>
</property>
</widget>
</item>
<item>
<widget class="QLineEdit" name="key"/>
</item>
<item>
<widget class="QPushButton" name="fetch">
<property name="text">
<string>Fetch</string>
</property>
</widget>
</item>
</layout>
</item>
<item>
<widget class="QLabel" name="warning">
<property name="text">
<string/>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<widget class="QGroupBox" name="groupBox">
<property name="title">
<string>Matches</string>
</property>
<layout class="QVBoxLayout">
<item>
<widget class="QLabel" name="label_3">
<property name="text">
<string>Select the book that most closely matches your copy from the list below</string>
</property>
</widget>
</item>
<item>
<widget class="QTableView" name="matches">
<property name="sizePolicy">
<sizepolicy hsizetype="Expanding" vsizetype="Expanding">
<horstretch>0</horstretch>
<verstretch>1</verstretch>
</sizepolicy>
</property>
<property name="alternatingRowColors">
<bool>true</bool>
</property>
<property name="selectionMode">
<enum>QAbstractItemView::SingleSelection</enum>
</property>
<property name="selectionBehavior">
<enum>QAbstractItemView::SelectRows</enum>
</property>
</widget>
</item>
<item>
<widget class="QTextBrowser" name="summary"/>
</item>
</layout>
</widget>
</item>
<item>
<widget class="QCheckBox" name="opt_get_social_metadata">
<property name="text">
<string>Download &amp;social metadata (tags/rating/etc.) for the selected book</string>
</property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="opt_overwrite_author_title_metadata">
<property name="text">
<string>Overwrite author and title with author and title of selected book</string>
</property>
</widget>
</item>
<item>
<widget class="QDialogButtonBox" name="buttonBox">
<property name="standardButtons">
<set>QDialogButtonBox::Cancel|QDialogButtonBox::Ok</set>
</property>
</widget>
</item>
</layout>
</widget>
<resources>
<include location="../../../../resources/images.qrc"/>
</resources>
<connections>
<connection>
<sender>buttonBox</sender>
<signal>accepted()</signal>
<receiver>FetchMetadata</receiver>
<slot>accept()</slot>
<hints>
<hint type="sourcelabel">
<x>460</x>
<y>599</y>
</hint>
<hint type="destinationlabel">
<x>657</x>
<y>530</y>
</hint>
</hints>
</connection>
<connection>
<sender>buttonBox</sender>
<signal>rejected()</signal>
<receiver>FetchMetadata</receiver>
<slot>reject()</slot>
<hints>
<hint type="sourcelabel">
<x>417</x>
<y>599</y>
</hint>
<hint type="destinationlabel">
<x>0</x>
<y>491</y>
</hint>
</hints>
</connection>
</connections>
</ui>
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>FetchMetadata</class>
<widget class="QDialog" name="FetchMetadata">
<property name="windowModality">
<enum>Qt::WindowModal</enum>
</property>
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>890</width>
<height>642</height>
</rect>
</property>
<property name="windowTitle">
<string>Fetch metadata</string>
</property>
<property name="windowIcon">
<iconset resource="../../../../resources/images.qrc">
<normaloff>:/images/metadata.png</normaloff>:/images/metadata.png</iconset>
</property>
<layout class="QVBoxLayout">
<item>
<widget class="QLabel" name="tlabel">
<property name="text">
<string>&lt;p&gt;calibre can find metadata for your books from two locations: &lt;b&gt;Google Books&lt;/b&gt; and &lt;b&gt;isbndb.com&lt;/b&gt;. &lt;p&gt;To use isbndb.com you must sign up for a &lt;a href=&quot;http://www.isbndb.com&quot;&gt;free account&lt;/a&gt; and enter your access key below.</string>
</property>
<property name="alignment">
<set>Qt::AlignCenter</set>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
<property name="openExternalLinks">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<layout class="QHBoxLayout">
<item>
<widget class="QLabel" name="label_2">
<property name="text">
<string>&amp;Access Key:</string>
</property>
<property name="buddy">
<cstring>key</cstring>
</property>
</widget>
</item>
<item>
<widget class="QLineEdit" name="key"/>
</item>
<item>
<widget class="QPushButton" name="fetch">
<property name="text">
<string>Fetch</string>
</property>
</widget>
</item>
</layout>
</item>
<item>
<widget class="QLabel" name="warning">
<property name="text">
<string/>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<widget class="QGroupBox" name="groupBox">
<property name="title">
<string>Matches</string>
</property>
<layout class="QVBoxLayout">
<item>
<widget class="QLabel" name="label_3">
<property name="text">
<string>Select the book that most closely matches your copy from the list below</string>
</property>
</widget>
</item>
<item>
<widget class="QTableView" name="matches">
<property name="sizePolicy">
<sizepolicy hsizetype="Expanding" vsizetype="Expanding">
<horstretch>0</horstretch>
<verstretch>1</verstretch>
</sizepolicy>
</property>
<property name="alternatingRowColors">
<bool>true</bool>
</property>
<property name="selectionMode">
<enum>QAbstractItemView::SingleSelection</enum>
</property>
<property name="selectionBehavior">
<enum>QAbstractItemView::SelectRows</enum>
</property>
</widget>
</item>
<item>
<widget class="QTextBrowser" name="summary"/>
</item>
</layout>
</widget>
</item>
<item>
<widget class="QCheckBox" name="opt_overwrite_author_title_metadata">
<property name="text">
<string>Overwrite author and title with author and title of selected book</string>
</property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="opt_get_social_metadata">
<property name="text">
<string>Download &amp;social metadata (tags/rating/etc.) for the selected book</string>
</property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="opt_auto_download_cover">
<property name="text">
<string>Automatically download the cover, if available</string>
</property>
</widget>
</item>
<item>
<widget class="QDialogButtonBox" name="buttonBox">
<property name="standardButtons">
<set>QDialogButtonBox::Cancel|QDialogButtonBox::Ok</set>
</property>
</widget>
</item>
</layout>
</widget>
<resources>
<include location="../../../../resources/images.qrc"/>
</resources>
<connections>
<connection>
<sender>buttonBox</sender>
<signal>accepted()</signal>
<receiver>FetchMetadata</receiver>
<slot>accept()</slot>
<hints>
<hint type="sourcelabel">
<x>460</x>
<y>599</y>
</hint>
<hint type="destinationlabel">
<x>657</x>
<y>530</y>
</hint>
</hints>
</connection>
<connection>
<sender>buttonBox</sender>
<signal>rejected()</signal>
<receiver>FetchMetadata</receiver>
<slot>reject()</slot>
<hints>
<hint type="sourcelabel">
<x>417</x>
<y>599</y>
</hint>
<hint type="destinationlabel">
<x>0</x>
<y>491</y>
</hint>
</hints>
</connection>
</connections>
</ui>

14
src/calibre/gui2/dialogs/metadata_bulk.py
View File

@ -17,6 +17,7 @@ from calibre.gui2 import error_dialog
from calibre.gui2.progress_indicator import ProgressIndicator
from calibre.utils.config import dynamic
from calibre.utils.titlecase import titlecase
from calibre.utils.icu import sort_key, capitalize
class MyBlockingBusy(QDialog):
@ -183,9 +184,10 @@ class MyBlockingBusy(QDialog):
class MetadataBulkDialog(QDialog, Ui_MetadataBulkDialog):
s_r_functions = { '' : lambda x: x,
_('Lower Case') : lambda x: x.lower(),
_('Upper Case') : lambda x: x.upper(),
_('Lower Case') : lambda x: icu_lower(x),
_('Upper Case') : lambda x: icu_upper(x),
_('Title Case') : lambda x: titlecase(x),
_('Capitalize') : lambda x: capitalize(x),
}
s_r_match_modes = [ _('Character match'),
@ -255,7 +257,7 @@ class MetadataBulkDialog(QDialog, Ui_MetadataBulkDialog):
fm = self.db.field_metadata
for f in fm:
if (f in ['author_sort'] or
(fm[f]['datatype'] in ['text', 'series']
(fm[f]['datatype'] in ['text', 'series', 'enumeration']
and fm[f].get('search_terms', None)
and f not in ['formats', 'ondevice', 'sort'])):
self.all_fields.append(f)
@ -594,7 +596,7 @@ class MetadataBulkDialog(QDialog, Ui_MetadataBulkDialog):
def initalize_authors(self):
all_authors = self.db.all_authors()
all_authors.sort(cmp=lambda x, y : cmp(x[1].lower(), y[1].lower()))
all_authors.sort(key=lambda x : sort_key(x[1]))
for i in all_authors:
id, name = i
@ -604,7 +606,7 @@ class MetadataBulkDialog(QDialog, Ui_MetadataBulkDialog):
def initialize_series(self):
all_series = self.db.all_series()
all_series.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_series.sort(key=lambda x : sort_key(x[1]))
for i in all_series:
id, name = i
@ -613,7 +615,7 @@ class MetadataBulkDialog(QDialog, Ui_MetadataBulkDialog):
def initialize_publisher(self):
all_publishers = self.db.all_publishers()
all_publishers.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_publishers.sort(key=lambda x : sort_key(x[1]))
for i in all_publishers:
id, name = i

324
src/calibre/gui2/dialogs/metadata_single.py
View File

@ -7,9 +7,12 @@ add/remove formats
'''
import os, re, time, traceback, textwrap
from functools import partial
from threading import Thread
from PyQt4.Qt import SIGNAL, QObject, Qt, QTimer, QThread, QDate, \
QPixmap, QListWidgetItem, QDialog, pyqtSignal, QMessageBox
from PyQt4.Qt import SIGNAL, QObject, Qt, QTimer, QDate, \
QPixmap, QListWidgetItem, QDialog, pyqtSignal, QMessageBox, QIcon, \
QPushButton
from calibre.gui2 import error_dialog, file_icon_provider, dynamic, \
choose_files, choose_images, ResizableDialog, \
@ -26,14 +29,18 @@ from calibre.ebooks.metadata.meta import get_metadata
from calibre.ebooks.metadata import MetaInformation
from calibre.utils.config import prefs, tweaks
from calibre.utils.date import qt_to_dt, local_tz, utcfromtimestamp
from calibre.utils.icu import sort_key
from calibre.customize.ui import run_plugins_on_import, get_isbndb_key
from calibre.gui2.preferences.social import SocialMetadata
from calibre.gui2.custom_column_widgets import populate_metadata_page
from calibre import strftime
class CoverFetcher(QThread):
class CoverFetcher(Thread): # {{{
def __init__(self, username, password, isbn, timeout, title, author):
Thread.__init__(self)
self.daemon = True
self.username = username.strip() if username else username
self.password = password.strip() if password else password
self.timeout = timeout
@ -41,8 +48,7 @@ class CoverFetcher(QThread):
self.title = title
self.needs_isbn = False
self.author = author
QThread.__init__(self)
self.exception = self.traceback = self.cover_data = None
self.exception = self.traceback = self.cover_data = self.errors = None
def run(self):
try:
@ -74,9 +80,9 @@ class CoverFetcher(QThread):
self.traceback = traceback.format_exc()
print self.traceback
# }}}
class Format(QListWidgetItem):
class Format(QListWidgetItem): # {{{
def __init__(self, parent, ext, size, path=None, timestamp=None):
self.path = path
@ -92,15 +98,70 @@ class Format(QListWidgetItem):
self.setToolTip(text)
self.setStatusTip(text)
# }}}
class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
COVER_FETCH_TIMEOUT = 240 # seconds
view_format = pyqtSignal(object)
# Cover processing {{{
def set_cover(self):
mi, ext = self.get_selected_format_metadata()
if mi is None:
return
cdata = None
if mi.cover and os.access(mi.cover, os.R_OK):
cdata = open(mi.cover).read()
elif mi.cover_data[1] is not None:
cdata = mi.cover_data[1]
if cdata is None:
error_dialog(self, _('Could not read cover'),
_('Could not read cover from %s format')%ext).exec_()
return
pix = QPixmap()
pix.loadFromData(cdata)
if pix.isNull():
error_dialog(self, _('Could not read cover'),
_('The cover in the %s format is invalid')%ext).exec_()
return
self.cover.setPixmap(pix)
self.update_cover_tooltip()
self.cover_changed = True
self.cpixmap = pix
self.cover_data = cdata
def trim_cover(self, *args):
from calibre.utils.magick import Image
cdata = self.cover_data
if not cdata:
return
im = Image()
im.load(cdata)
im.trim(10)
cdata = im.export('png')
pix = QPixmap()
pix.loadFromData(cdata)
self.cover.setPixmap(pix)
self.update_cover_tooltip()
self.cover_changed = True
self.cpixmap = pix
self.cover_data = cdata
def update_cover_tooltip(self):
p = self.cover.pixmap()
self.cover.setToolTip(_('Cover size: %dx%d pixels') %
(p.width(), p.height()))
def do_reset_cover(self, *args):
pix = QPixmap(I('default_cover.png'))
self.cover.setPixmap(pix)
self.update_cover_tooltip()
self.cover_changed = True
self.cover_data = None
@ -136,6 +197,7 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
else:
self.cover_path.setText(_file)
self.cover.setPixmap(pix)
self.update_cover_tooltip()
self.cover_changed = True
self.cpixmap = pix
self.cover_data = cover
@ -161,9 +223,80 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
pix = QPixmap()
pix.loadFromData(self.cover_data)
self.cover.setPixmap(pix)
self.update_cover_tooltip()
self.cover_changed = True
self.cpixmap = pix
def cover_dropped(self, cover_data):
self.cover_changed = True
self.cover_data = cover_data
self.update_cover_tooltip()
def fetch_cover(self):
isbn = re.sub(r'[^0-9a-zA-Z]', '', unicode(self.isbn.text())).strip()
self.fetch_cover_button.setEnabled(False)
self.setCursor(Qt.WaitCursor)
title, author = map(unicode, (self.title.text(), self.authors.text()))
self.cover_fetcher = CoverFetcher(None, None, isbn,
self.timeout, title, author)
self.cover_fetcher.start()
self._hangcheck = QTimer(self)
self._hangcheck.timeout.connect(self.hangcheck,
type=Qt.QueuedConnection)
self.cf_start_time = time.time()
self.pi.start(_('Downloading cover...'))
self._hangcheck.start(100)
def hangcheck(self):
if self.cover_fetcher.is_alive() and \
time.time()-self.cf_start_time < self.COVER_FETCH_TIMEOUT:
return
self._hangcheck.stop()
try:
if self.cover_fetcher.is_alive():
error_dialog(self, _('Cannot fetch cover'),
_('<b>Could not fetch cover.</b><br/>')+
_('The download timed out.')).exec_()
return
if self.cover_fetcher.needs_isbn:
error_dialog(self, _('Cannot fetch cover'),
_('Could not find cover for this book. Try '
'specifying the ISBN first.')).exec_()
return
if self.cover_fetcher.exception is not None:
err = self.cover_fetcher.exception
error_dialog(self, _('Cannot fetch cover'),
_('<b>Could not fetch cover.</b><br/>')+unicode(err)).exec_()
return
if self.cover_fetcher.errors and self.cover_fetcher.cover_data is None:
details = u'\n\n'.join([e[-1] + ': ' + e[1] for e in self.cover_fetcher.errors])
error_dialog(self, _('Cannot fetch cover'),
_('<b>Could not fetch cover.</b><br/>') +
_('For the error message from each cover source, '
'click Show details below.'), det_msg=details, show=True)
return
pix = QPixmap()
pix.loadFromData(self.cover_fetcher.cover_data)
if pix.isNull():
error_dialog(self, _('Bad cover'),
_('The cover is not a valid picture')).exec_()
else:
self.cover.setPixmap(pix)
self.update_cover_tooltip()
self.cover_changed = True
self.cpixmap = pix
self.cover_data = self.cover_fetcher.cover_data
finally:
self.fetch_cover_button.setEnabled(True)
self.unsetCursor()
self.pi.stop()
# }}}
# Formats processing {{{
def add_format(self, x):
files = choose_files(self, 'add formats dialog',
_("Choose formats for ") + unicode((self.title.text())),
@ -276,48 +409,6 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.comments.setPlainText(mi.comments)
def set_cover(self):
mi, ext = self.get_selected_format_metadata()
if mi is None:
return
cdata = None
if mi.cover and os.access(mi.cover, os.R_OK):
cdata = open(mi.cover).read()
elif mi.cover_data[1] is not None:
cdata = mi.cover_data[1]
if cdata is None:
error_dialog(self, _('Could not read cover'),
_('Could not read cover from %s format')%ext).exec_()
return
pix = QPixmap()
pix.loadFromData(cdata)
if pix.isNull():
error_dialog(self, _('Could not read cover'),
_('The cover in the %s format is invalid')%ext).exec_()
return
self.cover.setPixmap(pix)
self.cover_changed = True
self.cpixmap = pix
self.cover_data = cdata
def trim_cover(self, *args):
from calibre.utils.magick import Image
cdata = self.cover_data
if not cdata:
return
im = Image()
im.load(cdata)
im.trim(10)
cdata = im.export('png')
pix = QPixmap()
pix.loadFromData(cdata)
self.cover.setPixmap(pix)
self.cover_changed = True
self.cpixmap = pix
self.cover_data = cdata
def sync_formats(self):
old_extensions, new_extensions, paths = set(), set(), {}
for row in range(self.formats.count()):
@ -338,11 +429,14 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
if ext not in extensions:
self.db.remove_format(self.row, ext, notify=False)
def do_cancel_all(self):
self.cancel_all = True
self.reject()
def show_format(self, item, *args):
fmt = item.ext
self.view_format.emit(fmt)
def __init__(self, window, row, db, accepted_callback=None, cancel_all=False):
# }}}
def __init__(self, window, row, db, prev=None,
next_=None):
ResizableDialog.__init__(self, window)
self.bc_box.layout().setAlignment(self.cover, Qt.AlignCenter|Qt.AlignHCenter)
self.cancel_all = False
@ -354,16 +448,27 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
_(' The red color indicates that the current '
'author sort does not match the current author'))
if cancel_all:
self.__abort_button = self.button_box.addButton(self.button_box.Abort)
self.__abort_button.setToolTip(_('Abort the editing of all remaining books'))
self.connect(self.__abort_button, SIGNAL('clicked()'),
self.do_cancel_all)
self.row_delta = 0
if prev:
self.prev_button = QPushButton(QIcon(I('back.png')), _('Previous'),
self)
self.button_box.addButton(self.prev_button, self.button_box.ActionRole)
tip = _('Save changes and edit the metadata of %s')%prev
self.prev_button.setToolTip(tip)
self.prev_button.clicked.connect(partial(self.next_triggered,
-1))
if next_:
self.next_button = QPushButton(QIcon(I('forward.png')), _('Next'),
self)
self.button_box.addButton(self.next_button, self.button_box.ActionRole)
tip = _('Save changes and edit the metadata of %s')%next_
self.next_button.setToolTip(tip)
self.next_button.clicked.connect(partial(self.next_triggered, 1))
self.splitter.setStretchFactor(100, 1)
self.read_state()
self.db = db
self.pi = ProgressIndicator(self)
self.accepted_callback = accepted_callback
self.id = db.id(row)
self.row = row
self.cover_data = None
@ -412,6 +517,8 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.connect(self.reset_cover, SIGNAL('clicked()'), self.do_reset_cover)
self.connect(self.swap_button, SIGNAL('clicked()'), self.swap_title_author)
self.timeout = float(prefs['network_timeout'])
self.title.setText(db.title(row))
isbn = db.isbn(self.id, index_is_id=True)
if not isbn:
@ -472,6 +579,7 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
else:
self.cover_data = cover
self.cover.setPixmap(pm)
self.update_cover_tooltip()
self.original_series_name = unicode(self.series.text()).strip()
if len(db.custom_column_label_map) == 0:
self.central_widget.tabBar().setVisible(False)
@ -479,6 +587,9 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.create_custom_column_editors()
self.generate_cover_button.clicked.connect(self.generate_cover)
self.original_author = unicode(self.authors.text()).strip()
self.original_title = unicode(self.title.text()).strip()
def create_custom_column_editors(self):
w = self.central_widget.widget(1)
layout = w.layout()
@ -531,10 +642,6 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.isbn.setStyleSheet('QLineEdit { background-color: rgba(255,0,0,20%) }')
self.isbn.setToolTip(_('This ISBN number is invalid'))
def show_format(self, item, *args):
fmt = item.ext
self.view_format.emit(fmt)
def deduce_author_sort(self):
au = unicode(self.authors.text())
au = re.sub(r'\s+et al\.$', '', au)
@ -547,9 +654,6 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.authors.setText(title)
self.author_sort.setText('')
def cover_dropped(self, cover_data):
self.cover_changed = True
self.cover_data = cover_data
def initialize_combos(self):
self.initalize_authors()
@ -560,7 +664,7 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
def initalize_authors(self):
all_authors = self.db.all_authors()
all_authors.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_authors.sort(key=lambda x : sort_key(x[1]))
for i in all_authors:
id, name = i
name = [name.strip().replace('|', ',') for n in name.split(',')]
@ -575,7 +679,7 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
def initialize_series(self):
self.series.setSizeAdjustPolicy(self.series.AdjustToContentsOnFirstShow)
all_series = self.db.all_series()
all_series.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_series.sort(key=lambda x : sort_key(x[1]))
series_id = self.db.series_id(self.row)
idx, c = None, 0
for i in all_series:
@ -592,7 +696,7 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
def initialize_publisher(self):
all_publishers = self.db.all_publishers()
all_publishers.sort(cmp=lambda x, y : cmp(x[1], y[1]))
all_publishers.sort(key=lambda x : sort_key(x[1]))
publisher_id = self.db.publisher_id(self.row)
idx, c = None, 0
for i in all_publishers:
@ -625,66 +729,6 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.tags.setText(tag_string)
self.tags.update_tags_cache(self.db.all_tags())
def fetch_cover(self):
isbn = re.sub(r'[^0-9a-zA-Z]', '', unicode(self.isbn.text())).strip()
self.fetch_cover_button.setEnabled(False)
self.setCursor(Qt.WaitCursor)
title, author = map(unicode, (self.title.text(), self.authors.text()))
self.cover_fetcher = CoverFetcher(None, None, isbn,
self.timeout, title, author)
self.cover_fetcher.start()
self._hangcheck = QTimer(self)
self.connect(self._hangcheck, SIGNAL('timeout()'), self.hangcheck)
self.cf_start_time = time.time()
self.pi.start(_('Downloading cover...'))
self._hangcheck.start(100)
def hangcheck(self):
if not self.cover_fetcher.isFinished() and \
time.time()-self.cf_start_time < self.COVER_FETCH_TIMEOUT:
return
self._hangcheck.stop()
try:
if self.cover_fetcher.isRunning():
self.cover_fetcher.terminate()
error_dialog(self, _('Cannot fetch cover'),
_('<b>Could not fetch cover.</b><br/>')+
_('The download timed out.')).exec_()
return
if self.cover_fetcher.needs_isbn:
error_dialog(self, _('Cannot fetch cover'),
_('Could not find cover for this book. Try '
'specifying the ISBN first.')).exec_()
return
if self.cover_fetcher.exception is not None:
err = self.cover_fetcher.exception
error_dialog(self, _('Cannot fetch cover'),
_('<b>Could not fetch cover.</b><br/>')+unicode(err)).exec_()
return
if self.cover_fetcher.errors and self.cover_fetcher.cover_data is None:
details = u'\n\n'.join([e[-1] + ': ' + e[1] for e in self.cover_fetcher.errors])
error_dialog(self, _('Cannot fetch cover'),
_('<b>Could not fetch cover.</b><br/>') +
_('For the error message from each cover source, '
'click Show details below.'), det_msg=details, show=True)
return
pix = QPixmap()
pix.loadFromData(self.cover_fetcher.cover_data)
if pix.isNull():
error_dialog(self, _('Bad cover'),
_('The cover is not a valid picture')).exec_()
else:
self.cover.setPixmap(pix)
self.cover_changed = True
self.cpixmap = pix
self.cover_data = self.cover_fetcher.cover_data
finally:
self.fetch_cover_button.setEnabled(True)
self.unsetCursor()
self.pi.stop()
def fetch_metadata(self):
isbn = re.sub(r'[^0-9a-zA-Z]', '', unicode(self.isbn.text()))
@ -719,8 +763,8 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
if book.publisher: self.publisher.setEditText(book.publisher)
if book.isbn: self.isbn.setText(book.isbn)
if book.pubdate:
d = book.pubdate
self.pubdate.setDate(QDate(d.year, d.month, d.day))
dt = book.pubdate
self.pubdate.setDate(QDate(dt.year, dt.month, dt.day))
summ = book.comments
if summ:
prefix = unicode(self.comments.toPlainText())
@ -736,8 +780,11 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.series.setText(book.series)
if book.series_index is not None:
self.series_index.setValue(book.series_index)
# Needed because of Qt focus bug on OS X
self.fetch_cover_button.setFocus(Qt.OtherFocusReason)
if book.has_cover:
if d.opt_auto_download_cover.isChecked() and book.has_cover:
self.fetch_cover()
else:
self.fetch_cover_button.setFocus(Qt.OtherFocusReason)
else:
error_dialog(self, _('Cannot fetch metadata'),
_('You must specify at least one of ISBN, Title, '
@ -776,6 +823,10 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
unicode(self.tags.text()).split(',')],
notify=notify, commit=commit)
def next_triggered(self, row_delta, *args):
self.row_delta = row_delta
self.accept()
def accept(self):
cf = getattr(self, 'cover_fetcher', None)
if cf is not None and hasattr(cf, 'terminate'):
@ -785,9 +836,10 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
if self.formats_changed:
self.sync_formats()
title = unicode(self.title.text()).strip()
self.db.set_title(self.id, title, notify=False)
if title != self.original_title:
self.db.set_title(self.id, title, notify=False)
au = unicode(self.authors.text()).strip()
if au:
if au and au != self.original_author:
self.db.set_authors(self.id, string_to_authors(au), notify=False)
aus = unicode(self.author_sort.text()).strip()
if aus:
@ -837,8 +889,6 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
raise
self.save_state()
QDialog.accept(self)
if callable(self.accepted_callback):
self.accepted_callback(self.id)
def reject(self, *args):
cf = getattr(self, 'cover_fetcher', None)

3
src/calibre/gui2/dialogs/saved_search_editor.py
View File

@ -8,6 +8,7 @@ from PyQt4.QtGui import QDialog
from calibre.gui2.dialogs.saved_search_editor_ui import Ui_SavedSearchEditor
from calibre.utils.search_query_parser import saved_searches
from calibre.utils.icu import sort_key
from calibre.gui2.dialogs.confirm_delete import confirm
class SavedSearchEditor(QDialog, Ui_SavedSearchEditor):
@ -34,7 +35,7 @@ class SavedSearchEditor(QDialog, Ui_SavedSearchEditor):
def populate_search_list(self):
self.search_name_box.clear()
for name in sorted(self.searches.keys()):
for name in sorted(self.searches.keys(), key=sort_key):
self.search_name_box.addItem(name)
def add_search(self):

4
src/calibre/gui2/dialogs/search.py
View File

@ -8,6 +8,7 @@ from PyQt4.QtGui import QDialog, QDialogButtonBox
from calibre.gui2.dialogs.search_ui import Ui_Dialog
from calibre.library.caches import CONTAINS_MATCH, EQUALS_MATCH
from calibre.gui2 import gprefs
from calibre.utils.icu import sort_key
box_values = {}
@ -18,8 +19,7 @@ class SearchDialog(QDialog, Ui_Dialog):
self.setupUi(self)
self.mc = ''
searchables = sorted(db.field_metadata.searchable_fields(),
lambda x, y: cmp(x if x[0] != '#' else x[1:],
y if y[0] != '#' else y[1:]))
key=lambda x: sort_key(x if x[0] != '#' else x[1:]))
self.general_combo.addItems(searchables)
self.box_last_values = copy.deepcopy(box_values)

16
src/calibre/gui2/dialogs/tag_categories.py
View File

@ -9,6 +9,7 @@ from PyQt4.QtGui import QDialog, QIcon, QListWidgetItem
from calibre.gui2.dialogs.tag_categories_ui import Ui_TagCategories
from calibre.gui2.dialogs.confirm_delete import confirm
from calibre.constants import islinux
from calibre.utils.icu import sort_key
class Item:
def __init__(self, name, label, index, icon, exists):
@ -21,6 +22,15 @@ class Item:
return 'name=%s, label=%s, index=%s, exists='%(self.name, self.label, self.index, self.exists)
class TagCategories(QDialog, Ui_TagCategories):
'''
The structure of user_categories stored in preferences is
{cat_name: [ [name, category, v], [], []}, cat_name [ [name, cat, v] ...}
where name is the item name, category is where it came from (series, etc),
and v is a scratch area that this editor uses to keep track of categories.
If you add a category, it is permissible to set v to zero. If you delete
a category, ensure that both the name and the category match.
'''
category_labels_orig = ['', 'authors', 'series', 'publisher', 'tags']
def __init__(self, window, db, on_category=None):
@ -85,7 +95,7 @@ class TagCategories(QDialog, Ui_TagCategories):
# remove any references to a category that no longer exists
del self.categories[cat][item]
self.all_items_sorted = sorted(self.all_items, cmp=lambda x,y: cmp(x.name.lower(), y.name.lower()))
self.all_items_sorted = sorted(self.all_items, key=lambda x: sort_key(x.name))
self.display_filtered_categories(0)
for v in category_names:
@ -135,7 +145,7 @@ class TagCategories(QDialog, Ui_TagCategories):
index = self.all_items[node.data(Qt.UserRole).toPyObject()].index
if index not in self.applied_items:
self.applied_items.append(index)
self.applied_items.sort(cmp=lambda x, y:cmp(self.all_items[x].name.lower(), self.all_items[y].name.lower()))
self.applied_items.sort(key=lambda x:sort_key(self.all_items[x]))
self.display_filtered_categories(None)
def unapply_tags(self, node=None):
@ -198,5 +208,5 @@ class TagCategories(QDialog, Ui_TagCategories):
self.categories[self.current_cat_name] = l
def populate_category_list(self):
for n in sorted(self.categories.keys(), cmp=lambda x,y: cmp(x.lower(), y.lower())):
for n in sorted(self.categories.keys(), key=sort_key):
self.category_box.addItem(n)

16
src/calibre/gui2/dialogs/tag_editor.py
View File

@ -6,12 +6,10 @@ from PyQt4.QtGui import QDialog
from calibre.gui2.dialogs.tag_editor_ui import Ui_TagEditor
from calibre.gui2 import question_dialog, error_dialog
from calibre.constants import islinux
from calibre.utils.icu import sort_key
class TagEditor(QDialog, Ui_TagEditor):
def tag_cmp(self, x, y):
return cmp(x.lower(), y.lower())
def __init__(self, window, db, index=None):
QDialog.__init__(self, window)
Ui_TagEditor.__init__(self)
@ -25,7 +23,7 @@ class TagEditor(QDialog, Ui_TagEditor):
tags = []
if tags:
tags = [tag.strip() for tag in tags.split(',') if tag.strip()]
tags.sort(cmp=self.tag_cmp)
tags.sort(key=sort_key)
for tag in tags:
self.applied_tags.addItem(tag)
else:
@ -35,7 +33,7 @@ class TagEditor(QDialog, Ui_TagEditor):
all_tags = [tag for tag in self.db.all_tags()]
all_tags = list(set(all_tags))
all_tags.sort(cmp=self.tag_cmp)
all_tags.sort(key=sort_key)
for tag in all_tags:
if tag not in tags:
self.available_tags.addItem(tag)
@ -82,7 +80,7 @@ class TagEditor(QDialog, Ui_TagEditor):
self.tags.append(tag)
self.available_tags.takeItem(self.available_tags.row(item))
self.tags.sort(cmp=self.tag_cmp)
self.tags.sort(key=sort_key)
self.applied_tags.clear()
for tag in self.tags:
self.applied_tags.addItem(tag)
@ -96,14 +94,14 @@ class TagEditor(QDialog, Ui_TagEditor):
self.tags.remove(tag)
self.available_tags.addItem(tag)
self.tags.sort(cmp=self.tag_cmp)
self.tags.sort(key=sort_key)
self.applied_tags.clear()
for tag in self.tags:
self.applied_tags.addItem(tag)
items = [unicode(self.available_tags.item(x).text()) for x in
range(self.available_tags.count())]
items.sort(cmp=self.tag_cmp)
items.sort(key=sort_key)
self.available_tags.clear()
for item in items:
self.available_tags.addItem(item)
@ -117,7 +115,7 @@ class TagEditor(QDialog, Ui_TagEditor):
if tag not in self.tags:
self.tags.append(tag)
self.tags.sort(cmp=self.tag_cmp)
self.tags.sort(key=sort_key)
self.applied_tags.clear()
for tag in self.tags:
self.applied_tags.addItem(tag)

4
src/calibre/gui2/dialogs/tag_list_editor.py
View File

@ -39,7 +39,7 @@ class ListWidgetItem(QListWidgetItem):
class TagListEditor(QDialog, Ui_TagListEditor):
def __init__(self, window, tag_to_match, data, compare):
def __init__(self, window, tag_to_match, data, key):
QDialog.__init__(self, window)
Ui_TagListEditor.__init__(self)
self.setupUi(self)
@ -54,7 +54,7 @@ class TagListEditor(QDialog, Ui_TagListEditor):
for k,v in data:
self.all_tags[v] = k
for tag in sorted(self.all_tags.keys(), cmp=compare):
for tag in sorted(self.all_tags.keys(), key=key):
item = ListWidgetItem(tag)
item.setData(Qt.UserRole, self.all_tags[tag])
self.available_tags.addItem(item)

3
src/calibre/gui2/dialogs/user_profiles.py
View File

@ -13,6 +13,7 @@ from calibre.gui2 import error_dialog, question_dialog, open_url, \
choose_files, ResizableDialog, NONE
from calibre.gui2.widgets import PythonHighlighter
from calibre.ptempfile import PersistentTemporaryFile
from calibre.utils.icu import sort_key
class CustomRecipeModel(QAbstractListModel):
@ -256,7 +257,7 @@ class %(classname)s(%(base_class)s):
def add_builtin_recipe(self):
from calibre.web.feeds.recipes.collection import \
get_builtin_recipe_by_title, get_builtin_recipe_titles
items = sorted(get_builtin_recipe_titles())
items = sorted(get_builtin_recipe_titles(), key=sort_key)
title, ok = QInputDialog.getItem(self, _('Pick recipe'), _('Pick the recipe to customize'),

54
src/calibre/gui2/library/delegates.py
View File

@ -20,6 +20,7 @@ from calibre.gui2.widgets import EnLineEdit, TagsLineEdit
from calibre.utils.date import now, format_date
from calibre.utils.config import tweaks
from calibre.utils.formatter import validation_formatter
from calibre.utils.icu import sort_key
from calibre.gui2.dialogs.comments_dialog import CommentsDialog
class RatingDelegate(QStyledItemDelegate): # {{{
@ -173,7 +174,8 @@ class TagsDelegate(QStyledItemDelegate): # {{{
editor = TagsLineEdit(parent, self.db.all_tags())
else:
editor = TagsLineEdit(parent,
sorted(list(self.db.all_custom(label=self.db.field_metadata.key_to_label(col)))))
sorted(list(self.db.all_custom(label=self.db.field_metadata.key_to_label(col))),
key=sort_key))
return editor
else:
editor = EnLineEdit(parent)
@ -245,7 +247,8 @@ class CcTextDelegate(QStyledItemDelegate): # {{{
editor.setDecimals(2)
else:
editor = EnLineEdit(parent)
complete_items = sorted(list(m.db.all_custom(label=m.db.field_metadata.key_to_label(col))))
complete_items = sorted(list(m.db.all_custom(label=m.db.field_metadata.key_to_label(col))),
key=sort_key)
completer = QCompleter(complete_items, self)
completer.setCaseSensitivity(Qt.CaseInsensitive)
completer.setCompletionMode(QCompleter.PopupCompletion)
@ -254,6 +257,38 @@ class CcTextDelegate(QStyledItemDelegate): # {{{
# }}}
class CcEnumDelegate(QStyledItemDelegate): # {{{
'''
Delegate for text/int/float data.
'''
def createEditor(self, parent, option, index):
m = index.model()
col = m.column_map[index.column()]
editor = QComboBox(parent)
editor.addItem('')
for v in m.custom_columns[col]['display']['enum_values']:
editor.addItem(v)
return editor
def setModelData(self, editor, model, index):
val = unicode(editor.currentText())
if not val:
val = None
model.setData(index, QVariant(val), Qt.EditRole)
def setEditorData(self, editor, index):
m = index.model()
val = m.db.data[index.row()][m.custom_columns[m.column_map[index.column()]]['rec_index']]
if val is None:
val = ''
idx = editor.findText(val)
if idx < 0:
editor.setCurrentIndex(0)
else:
editor.setCurrentIndex(idx)
# }}}
class CcCommentsDelegate(QStyledItemDelegate): # {{{
'''
Delegate for comments data.
@ -314,10 +349,19 @@ class CcTemplateDelegate(QStyledItemDelegate): # {{{
QStyledItemDelegate.__init__(self, parent)
def createEditor(self, parent, option, index):
return EnLineEdit(parent)
m = index.model()
text = m.custom_columns[m.column_map[index.column()]]['display']['composite_template']
editor = CommentsDialog(parent, text)
editor.setWindowTitle(_("Edit template"))
editor.textbox.setTabChangesFocus(False)
editor.textbox.setTabStopWidth(20)
d = editor.exec_()
if d:
m.setData(index, QVariant(editor.textbox.toPlainText()), Qt.EditRole)
return None
def setModelData(self, editor, model, index):
val = unicode(editor.text())
val = unicode(editor.textbox.toPlainText())
try:
validation_formatter.validate(val)
except Exception, err:
@ -329,7 +373,7 @@ class CcTemplateDelegate(QStyledItemDelegate): # {{{
def setEditorData(self, editor, index):
m = index.model()
val = m.custom_columns[m.column_map[index.column()]]['display']['composite_template']
editor.setText(val)
editor.textbox.setPlainText(val)
# }}}

59
src/calibre/gui2/library/models.py
View File

@ -18,6 +18,7 @@ from calibre.ebooks.metadata import fmt_sidx, authors_to_string, string_to_autho
from calibre.ptempfile import PersistentTemporaryFile
from calibre.utils.config import tweaks, prefs
from calibre.utils.date import dt_factory, qt_to_dt, isoformat
from calibre.utils.icu import sort_key, strcmp as icu_strcmp
from calibre.ebooks.metadata.meta import set_metadata as _set_metadata
from calibre.utils.search_query_parser import SearchQueryParser
from calibre.library.caches import _match, CONTAINS_MATCH, EQUALS_MATCH, \
@ -222,21 +223,22 @@ class BooksModel(QAbstractTableModel): # {{{
def by_author(self):
return self.sorted_on[0] == 'authors'
def books_deleted(self):
self.count_changed()
self.clear_caches()
self.reset()
def delete_books(self, indices):
ids = map(self.id, indices)
for id in ids:
self.db.delete_book(id, notify=False)
self.count_changed()
self.clear_caches()
self.reset()
self.books_deleted()
return ids
def delete_books_by_id(self, ids):
for id in ids:
self.db.delete_book(id)
self.count_changed()
self.clear_caches()
self.reset()
self.books_deleted()
def books_added(self, num):
if num > 0:
@ -301,13 +303,28 @@ class BooksModel(QAbstractTableModel): # {{{
return self.rowCount(None)
def get_book_display_info(self, idx):
def custom_keys_to_display():
ans = getattr(self, '_custom_fields_in_book_info', None)
if ans is None:
cfkeys = set(self.db.custom_field_keys())
yes_fields = set(tweaks['book_details_will_display'])
no_fields = set(tweaks['book_details_wont_display'])
if '*' in yes_fields:
yes_fields = cfkeys
if '*' in no_fields:
no_fields = cfkeys
ans = frozenset(yes_fields - no_fields)
setattr(self, '_custom_fields_in_book_info', ans)
return ans
data = {}
cdata = self.cover(idx)
if cdata:
data['cover'] = cdata
tags = self.db.tags(idx)
tags = list(self.db.get_tags(self.db.id(idx)))
if tags:
tags = tags.replace(',', ', ')
tags.sort(key=sort_key)
tags = ', '.join(tags)
else:
tags = _('None')
data[_('Tags')] = tags
@ -331,7 +348,10 @@ class BooksModel(QAbstractTableModel): # {{{
_('Book %s of %s.')%\
(sidx, prepare_string_for_xml(series))
mi = self.db.get_metadata(idx)
cf_to_display = custom_keys_to_display()
for key in mi.custom_field_keys():
if key not in cf_to_display:
continue
name, val = mi.format_field(key)
if val:
data[name] = val
@ -544,7 +564,7 @@ class BooksModel(QAbstractTableModel): # {{{
def tags(r, idx=-1):
tags = self.db.data[r][idx]
if tags:
return QVariant(', '.join(sorted(tags.split(','))))
return QVariant(', '.join(sorted(tags.split(','), key=sort_key)))
return None
def series_type(r, idx=-1, siix=-1):
@ -595,7 +615,7 @@ class BooksModel(QAbstractTableModel): # {{{
def text_type(r, mult=False, idx=-1):
text = self.db.data[r][idx]
if text and mult:
return QVariant(', '.join(sorted(text.split('|'))))
return QVariant(', '.join(sorted(text.split('|'),key=sort_key)))
return QVariant(text)
def number_type(r, idx=-1):
@ -634,7 +654,7 @@ class BooksModel(QAbstractTableModel): # {{{
for col in self.custom_columns:
idx = self.custom_columns[col]['rec_index']
datatype = self.custom_columns[col]['datatype']
if datatype in ('text', 'comments', 'composite'):
if datatype in ('text', 'comments', 'composite', 'enumeration'):
self.dc[col] = functools.partial(text_type, idx=idx,
mult=self.custom_columns[col]['is_multiple'])
elif datatype in ('int', 'float'):
@ -722,7 +742,11 @@ class BooksModel(QAbstractTableModel): # {{{
if typ in ('text', 'comments'):
val = unicode(value.toString()).strip()
val = val if val else None
if typ == 'bool':
elif typ == 'enumeration':
val = unicode(value.toString()).strip()
if not val:
val = None
elif typ == 'bool':
val = value.toPyObject()
elif typ == 'rating':
val = value.toInt()[0]
@ -730,7 +754,7 @@ class BooksModel(QAbstractTableModel): # {{{
val *= 2
elif typ in ('int', 'float'):
val = unicode(value.toString()).strip()
if val is None or not val:
if not val:
val = None
elif typ == 'datetime':
val = value.toDate()
@ -1017,8 +1041,7 @@ class DeviceBooksModel(BooksModel): # {{{
x = ''
if y == None:
y = ''
x, y = x.strip().lower(), y.strip().lower()
return cmp(x, y)
return icu_strcmp(x.strip(), y.strip())
return _strcmp
def datecmp(x, y):
x = self.db[x].datetime
@ -1029,8 +1052,8 @@ class DeviceBooksModel(BooksModel): # {{{
x, y = int(self.db[x].size), int(self.db[y].size)
return cmp(x, y)
def tagscmp(x, y):
x = ','.join(sorted(getattr(self.db[x], 'device_collections', []))).lower()
y = ','.join(sorted(getattr(self.db[y], 'device_collections', []))).lower()
x = ','.join(sorted(getattr(self.db[x], 'device_collections', []),key=sort_key))
y = ','.join(sorted(getattr(self.db[y], 'device_collections', []),key=sort_key))
return cmp(x, y)
def libcmp(x, y):
x, y = self.db[x].in_library, self.db[y].in_library
@ -1207,7 +1230,7 @@ class DeviceBooksModel(BooksModel): # {{{
elif cname == 'collections':
tags = self.db[self.map[row]].device_collections
if tags:
tags.sort(cmp=lambda x,y: cmp(x.lower(), y.lower()))
tags.sort(key=sort_key)
return QVariant(', '.join(tags))
elif DEBUG and cname == 'inlibrary':
return QVariant(self.db[self.map[row]].in_library)

6
src/calibre/gui2/library/views.py
View File

@ -14,7 +14,8 @@ from PyQt4.Qt import QTableView, Qt, QAbstractItemView, QMenu, pyqtSignal, \
from calibre.gui2.library.delegates import RatingDelegate, PubDateDelegate, \
TextDelegate, DateDelegate, TagsDelegate, CcTextDelegate, \
CcBoolDelegate, CcCommentsDelegate, CcDateDelegate, CcTemplateDelegate
CcBoolDelegate, CcCommentsDelegate, CcDateDelegate, CcTemplateDelegate, \
CcEnumDelegate
from calibre.gui2.library.models import BooksModel, DeviceBooksModel
from calibre.utils.config import tweaks, prefs
from calibre.gui2 import error_dialog, gprefs
@ -76,6 +77,7 @@ class BooksView(QTableView): # {{{
self.publisher_delegate = TextDelegate(self)
self.text_delegate = TextDelegate(self)
self.cc_text_delegate = CcTextDelegate(self)
self.cc_enum_delegate = CcEnumDelegate(self)
self.cc_bool_delegate = CcBoolDelegate(self)
self.cc_comments_delegate = CcCommentsDelegate(self)
self.cc_template_delegate = CcTemplateDelegate(self)
@ -427,6 +429,8 @@ class BooksView(QTableView): # {{{
self.setItemDelegateForColumn(cm.index(colhead), self.rating_delegate)
elif cc['datatype'] == 'composite':
self.setItemDelegateForColumn(cm.index(colhead), self.cc_template_delegate)
elif cc['datatype'] == 'enumeration':
self.setItemDelegateForColumn(cm.index(colhead), self.cc_enum_delegate)
else:
dattr = colhead+'_delegate'
delegate = colhead if hasattr(self, dattr) else 'text'

4
src/calibre/gui2/preferences/behavior.py
View File

@ -19,6 +19,7 @@ from calibre.utils.search_query_parser import saved_searches
from calibre.ebooks import BOOK_EXTENSIONS
from calibre.ebooks.oeb.iterator import is_supported
from calibre.constants import iswindows
from calibre.utils.icu import sort_key
class ConfigWidget(ConfigWidgetBase, Ui_Form):
@ -45,8 +46,7 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
choices = [(x.upper(), x) for x in output_formats]
r('output_format', prefs, choices=choices)
restrictions = sorted(saved_searches().names(),
cmp=lambda x,y: cmp(x.lower(), y.lower()))
restrictions = sorted(saved_searches().names(), key=sort_key)
choices = [('', '')] + [(x, x) for x in restrictions]
r('gui_restriction', db.prefs, choices=choices)
r('new_book_tags', prefs, setting=CommaSeparatedList)

38
src/calibre/gui2/preferences/create_custom_column.py
View File

@ -27,18 +27,20 @@ class CreateCustomColumn(QDialog, Ui_QCreateCustomColumn):
3:{'datatype':'series',
'text':_('Text column for keeping series-like information'),
'is_multiple':False},
4:{'datatype':'datetime',
4:{'datatype':'enumeration',
'text':_('Text, but with a fixed set of permitted values'), 'is_multiple':False},
5:{'datatype':'datetime',
'text':_('Date'), 'is_multiple':False},
5:{'datatype':'float',
6:{'datatype':'float',
'text':_('Floating point numbers'), 'is_multiple':False},
6:{'datatype':'int',
7:{'datatype':'int',
'text':_('Integers'), 'is_multiple':False},
7:{'datatype':'rating',
8:{'datatype':'rating',
'text':_('Ratings, shown with stars'),
'is_multiple':False},
8:{'datatype':'bool',
9:{'datatype':'bool',
'text':_('Yes/No'), 'is_multiple':False},
9:{'datatype':'composite',
10:{'datatype':'composite',
'text':_('Column built from other columns'), 'is_multiple':False},
}
@ -59,6 +61,7 @@ class CreateCustomColumn(QDialog, Ui_QCreateCustomColumn):
self.editing_col = editing
self.standard_colheads = standard_colheads
self.standard_colnames = standard_colnames
self.column_type_box.setMaxVisibleItems(len(self.column_types))
for t in self.column_types:
self.column_type_box.addItem(self.column_types[t]['text'])
self.column_type_box.currentIndexChanged.connect(self.datatype_changed)
@ -91,6 +94,8 @@ class CreateCustomColumn(QDialog, Ui_QCreateCustomColumn):
self.date_format_box.setText(c['display'].get('date_format', ''))
elif ct == 'composite':
self.composite_box.setText(c['display'].get('composite_template', ''))
elif ct == 'enumeration':
self.enum_box.setText(','.join(c['display'].get('enum_values', [])))
self.datatype_changed()
self.exec_()
@ -103,7 +108,8 @@ class CreateCustomColumn(QDialog, Ui_QCreateCustomColumn):
getattr(self, 'date_format_'+x).setVisible(col_type == 'datetime')
for x in ('box', 'default_label', 'label'):
getattr(self, 'composite_'+x).setVisible(col_type == 'composite')
for x in ('box', 'default_label', 'label'):
getattr(self, 'enum_'+x).setVisible(col_type == 'enumeration')
def accept(self):
col = unicode(self.column_name_box.text())
@ -145,17 +151,31 @@ class CreateCustomColumn(QDialog, Ui_QCreateCustomColumn):
return self.simple_error('', _('The heading %s is already used')%col_heading)
display_dict = {}
if col_type == 'datetime':
if self.date_format_box.text():
display_dict = {'date_format':unicode(self.date_format_box.text())}
else:
display_dict = {'date_format': None}
if col_type == 'composite':
elif col_type == 'composite':
if not self.composite_box.text():
return self.simple_error('', _('You must enter a template for'
' composite columns'))
display_dict = {'composite_template':unicode(self.composite_box.text())}
elif col_type == 'enumeration':
if not self.enum_box.text():
return self.simple_error('', _('You must enter at least one'
' value for enumeration columns'))
l = [v.strip() for v in unicode(self.enum_box.text()).split(',')]
for v in l:
if not v:
return self.simple_error('', _('You cannot provide the empty '
'value, as it is included by default'))
for i in range(0, len(l)-1):
if l[i] in l[i+1:]:
return self.simple_error('', _('The value "{0}" is in the '
'list more than once').format(l[i]))
display_dict = {'enum_values': l}
db = self.parent.gui.library_view.model().db
key = db.field_metadata.custom_field_prefix+col

59
src/calibre/gui2/preferences/create_custom_column.ui
View File

@ -10,7 +10,7 @@
<x>0</x>
<y>0</y>
<width>528</width>
<height>199</height>
<height>212</height>
</rect>
</property>
<property name="sizePolicy">
@ -24,7 +24,7 @@
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<layout class="QGridLayout" name="gridLayout_2" rowstretch="0,0,0,0">
<layout class="QGridLayout" name="gridLayout_2" rowstretch="0,0,0,0,0,0,0,0,0,0,0,0">
<property name="sizeConstraint">
<enum>QLayout::SetDefaultConstraint</enum>
</property>
@ -56,7 +56,7 @@
</property>
</widget>
</item>
<item row="0" column="1">
<item row="0" column="2">
<widget class="QLineEdit" name="column_name_box">
<property name="minimumSize">
<size>
@ -69,7 +69,7 @@
</property>
</widget>
</item>
<item row="1" column="1">
<item row="1" column="2">
<widget class="QLineEdit" name="column_heading_box">
<property name="toolTip">
<string>Column heading in the library view and category name in the tag browser</string>
@ -86,7 +86,7 @@
</property>
</widget>
</item>
<item row="2" column="1">
<item row="2" column="2">
<widget class="QComboBox" name="column_type_box">
<property name="sizePolicy">
<sizepolicy hsizetype="Maximum" vsizetype="Fixed">
@ -105,7 +105,7 @@
</property>
</widget>
</item>
<item row="4" column="1">
<item row="4" column="2">
<layout class="QHBoxLayout" name="horizontalLayout_3">
<item>
<widget class="QLineEdit" name="date_format_box">
@ -147,18 +147,18 @@
</property>
</widget>
</item>
<item row="5" column="1">
<item row="5" column="2">
<layout class="QHBoxLayout" name="horizontalLayout_4">
<item>
<widget class="QLineEdit" name="composite_box">
<property name="sizePolicy">
<sizepolicy hsizetype="Maximum" vsizetype="Fixed">
<sizepolicy hsizetype="Expanding" vsizetype="Fixed">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="toolTip">
<string>&lt;p&gt;Field template. Uses the same syntax as save templates.</string>
<string>Field template. Uses the same syntax as save templates.</string>
</property>
</widget>
</item>
@ -184,7 +184,7 @@
</property>
</widget>
</item>
<item row="10" column="0" colspan="3">
<item row="11" column="0" colspan="4">
<spacer name="verticalSpacer_2">
<property name="orientation">
<enum>Qt::Vertical</enum>
@ -197,6 +197,45 @@
</property>
</spacer>
</item>
<item row="6" column="0">
<widget class="QLabel" name="enum_label">
<property name="text">
<string>Values</string>
</property>
<property name="buddy">
<cstring>enum_box</cstring>
</property>
</widget>
</item>
<item row="6" column="2">
<layout class="QHBoxLayout" name="horizontalLayout_2">
<item>
<widget class="QLineEdit" name="enum_box">
<property name="sizePolicy">
<sizepolicy hsizetype="Expanding" vsizetype="Fixed">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="toolTip">
<string>A comma-separated list of permitted values. The empty value is always
included, and is the default. For example, the list 'one,two,three' has
four values, the first of them being the empty value.</string>
</property>
</widget>
</item>
<item>
<widget class="QLabel" name="enum_default_label">
<property name="toolTip">
<string>The empty string is always the first value</string>
</property>
<property name="text">
<string>Default: (nothing)</string>
</property>
</widget>
</item>
</layout>
</item>
</layout>
</item>
<item row="11" column="0">

1
src/calibre/gui2/preferences/plugins.py
View File

@ -151,6 +151,7 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
self._plugin_model.populate()
self._plugin_model.reset()
self.changed_signal.emit()
self.plugin_path.setText('')
else:
error_dialog(self, _('No valid plugin path'),
_('%s is not a valid plugin path')%path).exec_()

12
src/calibre/gui2/search_box.py
View File

@ -17,6 +17,7 @@ from calibre.gui2.dialogs.confirm_delete import confirm
from calibre.gui2.dialogs.saved_search_editor import SavedSearchEditor
from calibre.gui2.dialogs.search import SearchDialog
from calibre.utils.search_query_parser import saved_searches
from calibre.utils.icu import sort_key
class SearchLineEdit(QLineEdit): # {{{
key_pressed = pyqtSignal(object)
@ -204,7 +205,7 @@ class SearchBox2(QComboBox): # {{{
self.blockSignals(yes)
self.line_edit.blockSignals(yes)
def set_search_string(self, txt, store_in_history=False):
def set_search_string(self, txt, store_in_history=False, emit_changed=True):
self.setFocus(Qt.OtherFocusReason)
if not txt:
self.clear()
@ -212,7 +213,8 @@ class SearchBox2(QComboBox): # {{{
self.normalize_state()
self.setEditText(txt)
self.line_edit.end(False)
self.changed.emit()
if emit_changed:
self.changed.emit()
self._do_search(store_in_history=store_in_history)
self.focus_to_library.emit()
@ -292,7 +294,7 @@ class SavedSearchBox(QComboBox): # {{{
self.search_box.clear()
self.setEditText(qname)
return
self.search_box.set_search_string(u'search:"%s"' % qname)
self.search_box.set_search_string(u'search:"%s"' % qname, emit_changed=False)
self.setEditText(qname)
self.setToolTip(saved_searches().lookup(qname))
@ -384,7 +386,7 @@ class SearchBoxMixin(object): # {{{
def do_advanced_search(self, *args):
d = SearchDialog(self, self.library_view.model().db)
if d.exec_() == QDialog.Accepted:
self.search.set_search_string(d.search_string())
self.search.set_search_string(d.search_string(), store_in_history=True)
def do_search_button(self):
self.search.do_search()
@ -417,7 +419,7 @@ class SavedSearchBoxMixin(object): # {{{
b.setStatusTip(b.toolTip())
def saved_searches_changed(self):
p = sorted(saved_searches().names(), cmp=lambda x,y: cmp(x.lower(), y.lower()))
p = sorted(saved_searches().names(), key=sort_key)
t = unicode(self.search_restriction.currentText())
# rebuild the restrictions combobox using current saved searches
self.search_restriction.clear()

4
src/calibre/gui2/shortcuts.py
View File

@ -14,6 +14,7 @@ from PyQt4.Qt import QAbstractListModel, Qt, QKeySequence, QListView, \
from calibre.gui2 import NONE, error_dialog
from calibre.utils.config import XMLConfig
from calibre.utils.icu import sort_key
from calibre.gui2.shortcuts_ui import Ui_Frame
DEFAULTS = Qt.UserRole
@ -175,8 +176,7 @@ class Shortcuts(QAbstractListModel):
for k, v in shortcuts.items():
self.keys[k] = v[0]
self.order = list(shortcuts)
self.order.sort(cmp=lambda x,y : cmp(self.descriptions[x],
self.descriptions[y]))
self.order.sort(key=lambda x : sort_key(self.descriptions[x]))
self.sequences = {}
for k, v in self.keys.items():
self.sequences[k] = [QKeySequence(x) for x in v]

29
src/calibre/gui2/tag_view.py
View File

@ -18,6 +18,7 @@ from PyQt4.Qt import Qt, QTreeView, QApplication, pyqtSignal, \
from calibre.ebooks.metadata import title_sort
from calibre.gui2 import config, NONE
from calibre.library.field_metadata import TagsIcons, category_icon_map
from calibre.utils.icu import sort_key
from calibre.utils.search_query_parser import saved_searches
from calibre.gui2 import error_dialog
from calibre.gui2.dialogs.confirm_delete import confirm
@ -72,6 +73,7 @@ class TagsView(QTreeView): # {{{
def __init__(self, parent=None):
QTreeView.__init__(self, parent=None)
self.tag_match = None
self.disable_recounting = False
self.setUniformRowHeights(True)
self.setCursor(Qt.PointingHandCursor)
self.setIconSize(QSize(30, 30))
@ -106,10 +108,13 @@ class TagsView(QTreeView): # {{{
self.refresh_required.connect(self.recount, type=Qt.QueuedConnection)
self.sort_by.currentIndexChanged.connect(self.sort_changed)
self.made_connections = True
self.refresh_signal_processed = True
db.add_listener(self.database_changed)
def database_changed(self, event, ids):
self.refresh_required.emit()
if self.refresh_signal_processed:
self.refresh_signal_processed = False
self.refresh_required.emit()
@property
def match_all(self):
@ -222,7 +227,7 @@ class TagsView(QTreeView): # {{{
partial(self.context_menu_handler, action='hide', category=category))
if self.hidden_categories:
m = self.context_menu.addMenu(_('Show category'))
for col in sorted(self.hidden_categories, cmp=lambda x,y: cmp(x.lower(), y.lower())):
for col in sorted(self.hidden_categories, key=sort_key):
m.addAction(col,
partial(self.context_menu_handler, action='show', category=col))
@ -295,6 +300,9 @@ class TagsView(QTreeView): # {{{
return self.isExpanded(idx)
def recount(self, *args):
if self.disable_recounting:
return
self.refresh_signal_processed = True
ci = self.currentIndex()
if not ci.isValid():
ci = self.indexAt(QPoint(10, 10))
@ -595,7 +603,8 @@ class TagsModel(QAbstractItemModel): # {{{
# Reconstruct the user categories, putting them into metadata
self.db.field_metadata.remove_dynamic_categories()
tb_cats = self.db.field_metadata
for user_cat in sorted(self.db.prefs.get('user_categories', {}).keys()):
for user_cat in sorted(self.db.prefs.get('user_categories', {}).keys(),
key=sort_key):
cat_name = user_cat+':' # add the ':' to avoid name collision
tb_cats.add_user_category(label=cat_name, name=user_cat)
if len(saved_searches().names()):
@ -874,13 +883,13 @@ class TagBrowserMixin(object): # {{{
db=self.library_view.model().db
if category == 'tags':
result = db.get_tags_with_ids()
compare = (lambda x,y:cmp(x.lower(), y.lower()))
key = sort_key
elif category == 'series':
result = db.get_series_with_ids()
compare = (lambda x,y:cmp(title_sort(x).lower(), title_sort(y).lower()))
key = lambda x:sort_key(title_sort(x))
elif category == 'publisher':
result = db.get_publishers_with_ids()
compare = (lambda x,y:cmp(x.lower(), y.lower()))
key = sort_key
else: # should be a custom field
cc_label = None
if category in db.field_metadata:
@ -888,9 +897,9 @@ class TagBrowserMixin(object): # {{{
result = db.get_custom_items_with_ids(label=cc_label)
else:
result = []
compare = (lambda x,y:cmp(x.lower(), y.lower()))
key = sort_key
d = TagListEditor(self, tag_to_match=tag, data=result, compare=compare)
d = TagListEditor(self, tag_to_match=tag, data=result, key=key)
d.exec_()
if d.result() == d.Accepted:
to_rename = d.to_rename # dict of new text to old id
@ -937,7 +946,9 @@ class TagBrowserMixin(object): # {{{
if old_author != new_author:
# The id might change if the new author already exists
id = db.rename_author(id, new_author)
db.set_sort_field_for_author(id, unicode(new_sort))
db.set_sort_field_for_author(id, unicode(new_sort),
commit=False, notify=False)
db.commit()
self.library_view.model().refresh()
self.tags_view.recount()

Some files were not shown because too many files have changed in this diff Show More