mirror of
https://github.com/kovidgoyal/calibre.git
synced 2025-07-09 03:04:10 -04:00
merge from trunk
This commit is contained in:
ldolse
commit
704e266f0c
158
Changelog.yaml
158
Changelog.yaml
@ -4,6 +4,164 @@
|
||||
# for important features/bug fixes.
|
||||
# Also, each release can have new and improved recipes.
|
||||
|
||||
- version: 0.7.22
|
||||
date: 2010-10-03
|
||||
|
||||
new features:
|
||||
- title: "Drag and drop books from your calibre library"
|
||||
type: major
|
||||
description: >
|
||||
"You can now drag and drop books from your calibre library. You can drag them to the desktop or to a file explorer, to copy them to your computer. You can drag them to the
|
||||
device icon in calibre to send them to the device. You can also drag and drop books from the device view in calibre to the calibre library icon or the operating
|
||||
system to copy them from the device."
|
||||
|
||||
- title: "There were many minor bug fixes for various bugs caused by the major changes in 0.7.21. So if you have updated to 0.7.21, it is highly recommended you update to 0.7.22"
|
||||
|
||||
- title: "Driver for the VelocityMicro ebook reader device"
|
||||
|
||||
- title: "Add a tweak to control how articles in titles are processed during sorting"
|
||||
|
||||
- title: "Add a new format type 'device_db' to plugboards to control the metadata displayed in book lists on SONY devices."
|
||||
|
||||
bug fixes:
|
||||
- title: "Fix ISBN not being read from filenames in 0.7.21"
|
||||
tickets: [7054]
|
||||
|
||||
- title: "Fix instant Search for text not found causes unhandled exception when conversion jobs are running"
|
||||
tickets: [7043]
|
||||
|
||||
- title: "Fix removing a publisher causes an error in 0.7.21"
|
||||
tickets: [7046]
|
||||
|
||||
- title: "MOBI Output: Fix some images being distorted in 0.7.21"
|
||||
tickets: [7049]
|
||||
|
||||
- title: "Fix regression that broke bulk conversion of books without covers in 0.7.21"
|
||||
|
||||
- title: "Fix regression that broke add and set_metadata commands in calibredb in 0.7.21"
|
||||
|
||||
- title: "Workaround for Qt bug in file open dialogs in linux that causes multiple file selection to ignore files with two or more spaces in the file name"
|
||||
|
||||
- title: "Conversion pipeline: Fix regression in 0.7.21 that broke conversion of LIT/EPUB documents that specified no title in their OPF files"
|
||||
|
||||
- title: "Fix regression that broke iPad driver in 0.7.21"
|
||||
|
||||
improved recipes:
|
||||
- Washington Post
|
||||
|
||||
|
||||
- version: 0.7.21
|
||||
date: 2010-10-01
|
||||
|
||||
new features:
|
||||
- title: "Automatic backup of the calibre metadata database"
|
||||
type: major
|
||||
description: >
|
||||
"calibre now automatically backups up the metadata for each book in the library into an individual OPF file in that books' folder. This means that if the calibre metadata database is corrupted, for example by a hard disk failure, you can reconstruct it from these OPF files, without losing any metadata. For the moment, only the backup is implemented, restore will be implemented in the future. The backup happens automatically in the background while calibre is running. The first time you start calibre, all the books will need to be backed up, so you may notice calibre running a little slower than usual."
|
||||
|
||||
- title: "Virtual columns"
|
||||
type: major
|
||||
description: >
|
||||
"You can now add virtual columns to the calibre book list. These are built fro other columns using templates and can be used to, for example, create columns to show the books isbn and avaialbale formats. You can do this by right clicking on a column header and select 'Add your own columns'"
|
||||
|
||||
- title: "calibre templates now much more powerful"
|
||||
type: major
|
||||
description: >
|
||||
"The templates used in calibre in send to device and save to disk have now beome much ore powerful. They can use conditinal text and functions to transforms the replacement text. Also they now have access t metadata in user defined columns. For details see the tutorials section of the User Manual."
|
||||
|
||||
- title: "Metadata plugboards: Allow you to perform sophisticated transformations on the metadata of a book when exporting it from the calibre library."
|
||||
type: major
|
||||
description: >
|
||||
"For example, you can add the series informtion to the title when sendig books to a device. This functionality is accessed from Preferences->Import/Export->Metadata plugboards"
|
||||
|
||||
- title: "User defined columns are now fully integrated into calibre"
|
||||
type: major
|
||||
description: >
|
||||
"User defined columns can nw be used everywhere. In the content server, Search and Replace, to create ondevice collections, and in the save to disk and send to device templates for creating filenames. In addition, user defined metadata is saved to an read back from EPUB/OPF files."
|
||||
|
||||
- title: "Driver for the jetBook Mini"
|
||||
|
||||
- title: "Add tweaks to control which custom columns the content server displays."
|
||||
|
||||
- title: "Bulk downloading of metadata/covers now shows progress and can be canceled"
|
||||
|
||||
- title: "New plugin to download covers from douban.com. It is disabled by default and must be enabled via Preferences->Advanced->Plugins->Cover download plugins"
|
||||
|
||||
- title: "Add option to change titles to title case in the Bulk metadata edit dialog"
|
||||
|
||||
- title: "Add option to bulk metadata edit dialog to force series renumbering to start with a specified value"
|
||||
|
||||
bug fixes:
|
||||
- title: "Fix various bugs that could lead to stale files being left in the calbre library when editing title/author metadata on windows"
|
||||
|
||||
- title: "Fix various regression in the preprocess and de-hyphenation code that broke conversion of some files, especially PDF ones."
|
||||
|
||||
- title: "Alex driver: Fix books not being placed in sub directories. Send covers. And allow sending of FB2"
|
||||
tickets: [6956]
|
||||
|
||||
- title: "MOBI Output: Fix bug that could caused left margins in the MOBI file to have twice the size of the left margins in the input document, when viewed on the pathetic Kindle MOBI renderer"
|
||||
|
||||
- title: "MOBI Input: Interpret blockquotes as having a left margin of 2em not 1em to reflect recent Amazon practice"
|
||||
|
||||
- title: "MOBI Output: Remove transparencies from images. Pathetic Kindle MOBI renderer strikes again"
|
||||
|
||||
- title: "Revert removal of inline toc from news downloaded in MOBI format as this makes it unusable with the pathetic Kindle For PC application"
|
||||
|
||||
- title: "Content server: Remove special characters from filenames in download links to accomodate broken browsers like the one in the Kindle"
|
||||
|
||||
- title: "Conversion pipeline: When rescaling images, dont replace gif image data with jpeg data"
|
||||
|
||||
- title: "EPUB Input: Ignore OPF files in the EPUB whose names start with a period"
|
||||
|
||||
- title: "RTF Output: Handle a larger set of broken images in the input document"
|
||||
tickets: [7003]
|
||||
|
||||
- title: "epub-fix: Handle dates before 1900"
|
||||
tickets: [7002]
|
||||
|
||||
- title: "Welcome wizard: Prevent the user from choosing a non empty folder as her calibre library"
|
||||
|
||||
- title: "Automatically enable the Douban metadata download plugins if the user choose chinese as the interface language in the welcome wizard"
|
||||
|
||||
- title: "Linux DBUS notifier: Fix causing freezes on some DBUS implementations"
|
||||
tickets: [6969]
|
||||
|
||||
- title: "Workaround for windows limitation when reading from network sockets. Should fix issues with large files in calibre libraries on network shares."
|
||||
tickets: [3248]
|
||||
|
||||
new recipes:
|
||||
- title: "BBC Sport"
|
||||
author: "limawhiskey"
|
||||
|
||||
- title: "Revista Muy Interesante "
|
||||
author: "Jefferson Frantz"
|
||||
|
||||
- title: "El Universo - Ecuador and Frederik Pohl's Blog"
|
||||
author: "Darko Miletic"
|
||||
|
||||
- title: "Science News"
|
||||
author: "Starson17"
|
||||
|
||||
- title: "Various Belgian news sources"
|
||||
author: "Lionel Bergeret"
|
||||
|
||||
- title: "Oriental Daily"
|
||||
author: "Larry Chan"
|
||||
|
||||
- title: "Rmf24 - Opinie"
|
||||
author: "Tomasz Dlugosz"
|
||||
|
||||
- title: "Jerusalem Post - French and Howto Geek"
|
||||
author: "Tony Stegall"
|
||||
|
||||
|
||||
improved recipes:
|
||||
- Peter Schiff
|
||||
- Telegraph UK
|
||||
- AJC
|
||||
- Boortz
|
||||
- Scientific American
|
||||
|
||||
- version: 0.7.20
|
||||
date: 2010-09-24
|
||||
|
||||
|
||||
@ -83,6 +83,16 @@ title_series_sorting = 'library_order'
|
||||
# strictly_alphabetic, it would remain "The Client".
|
||||
save_template_title_series_sorting = 'library_order'
|
||||
|
||||
# Set the list of words that are to be considered 'articles' when computing the
|
||||
# title sort strings. The list is a regular expression, with the articles
|
||||
# separated by 'or' bars. Comparisons are case insensitive, and that cannot be
|
||||
# changed. Changes to this tweak won't have an effect until the book is modified
|
||||
# in some way. If you enter an invalid pattern, it is silently ignored.
|
||||
# To disable use the expression: '^$'
|
||||
# Default: '^(A|The|An)\s+'
|
||||
title_sort_articles=r'^(A|The|An)\s+'
|
||||
|
||||
|
||||
# Specify a folder that calibre should connect to at startup using
|
||||
# connect_to_folder. This must be a full path to the folder. If the folder does
|
||||
# not exist when calibre starts, it is ignored. If there are '\' characters in
|
||||
|
||||
BIN
resources/images/news/eluniverso_ec.png
Normal file
BIN
resources/images/news/eluniverso_ec.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 625 B |
BIN
resources/images/news/science_news_recent_issues.png
Normal file
BIN
resources/images/news/science_news_recent_issues.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 696 B |
65
resources/recipes/bbc_sport.recipe
Normal file
65
resources/recipes/bbc_sport.recipe
Normal file
@ -0,0 +1,65 @@
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2010, limawhiskey <limawhiskey at gmail.com>'
|
||||
'''
|
||||
news.bbc.co.uk/sport/
|
||||
'''
|
||||
import re
|
||||
from calibre.web.feeds.recipes import BasicNewsRecipe
|
||||
|
||||
class BBC(BasicNewsRecipe):
|
||||
title = 'BBC Sport'
|
||||
__author__ = 'limawhiskey, Darko Miletic, Starson17'
|
||||
description = 'Sports news from UK. A fast version that does not download pictures'
|
||||
oldest_article = 2
|
||||
max_articles_per_feed = 100
|
||||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
encoding = 'utf8'
|
||||
publisher = 'BBC'
|
||||
category = 'sport, news, UK, world'
|
||||
language = 'en_GB'
|
||||
publication_type = 'newsportal'
|
||||
extra_css = ' body{ font-family: Verdana,Helvetica,Arial,sans-serif } .introduction{font-weight: bold} .story-feature{display: block; padding: 0; border: 1px solid; width: 40%; font-size: small} .story-feature h2{text-align: center; text-transform: uppercase} '
|
||||
preprocess_regexps = [(re.compile(r'<!--.*?-->', re.DOTALL), lambda m: '')]
|
||||
conversion_options = {
|
||||
'comments' : description
|
||||
,'tags' : category
|
||||
,'language' : language
|
||||
,'publisher' : publisher
|
||||
,'linearize_tables': True
|
||||
}
|
||||
|
||||
keep_only_tags = [
|
||||
dict(name='div', attrs={'class':['ds','mxb']}),
|
||||
dict(attrs={'class':['story-body','storybody']})
|
||||
]
|
||||
|
||||
remove_tags = [
|
||||
dict(name='div', attrs={'class':['storyextra', 'share-help', 'embedded-hyper', \
|
||||
'story-feature wide ', 'story-feature narrow', 'cap', 'caption', 'q1', 'sihf', \
|
||||
'mva', 'videoInStoryC', 'sharesb', 'mvtb']}),
|
||||
dict(name=['img']), dict(name=['br'])
|
||||
]
|
||||
|
||||
remove_attributes = ['width','height']
|
||||
|
||||
feeds = [
|
||||
('Sport Front Page', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/front_page/rss.xml'),
|
||||
('Football', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/football/rss.xml'),
|
||||
('Cricket', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/cricket/rss.xml'),
|
||||
('Formula 1', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/motorsport/formula_one/rss.xml'),
|
||||
('Commonwealth Games', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/commonwealth_games/delhi_2010/rss.xml'),
|
||||
('Golf', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/golf/rss.xml'),
|
||||
('Rugby Union', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/rugby_union/rss.xml'),
|
||||
('Rugby League', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/rugby_league/rss.xml'),
|
||||
('Tennis', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/tennis/rss.xml'),
|
||||
('Motorsport', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/motorsport/rss.xml'),
|
||||
('Boxing', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/boxing/rss.xml'),
|
||||
('Athletics', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/athletics/rss.xml'),
|
||||
('Snooker', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/other_sports/snooker/rss.xml'),
|
||||
('Horse Racing', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/other_sports/horse_racing/rss.xml'),
|
||||
('Cycling', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/other_sports/cycling/rss.xml'),
|
||||
('Disability Sport', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/other_sports/disability_sport/rss.xml'),
|
||||
('Other Sport', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/other_sports/rss.xml'),
|
||||
('Olympics 2012', 'http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/olympics/london_2012/rss.xml'),
|
||||
]
|
||||
63
resources/recipes/eluniverso_ec.recipe
Normal file
63
resources/recipes/eluniverso_ec.recipe
Normal file
@ -0,0 +1,63 @@
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
|
||||
'''
|
||||
eluniverso.com
|
||||
'''
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
class ElUniverso_Ecuador(BasicNewsRecipe):
|
||||
title = 'El Universo - Ecuador'
|
||||
__author__ = 'Darko Miletic'
|
||||
description = 'Noticias del Ecuador y el resto del mundo'
|
||||
publisher = 'El Universo'
|
||||
category = 'news, politics, Ecuador'
|
||||
oldest_article = 2
|
||||
max_articles_per_feed = 200
|
||||
no_stylesheets = True
|
||||
encoding = 'utf8'
|
||||
use_embedded_content = False
|
||||
language = 'es'
|
||||
remove_empty_feeds = True
|
||||
publication_type = 'newspaper'
|
||||
masthead_url = 'http://servicios2.eluniverso.com/versiones/v1/img/Hd/lg_ElUniverso.gif'
|
||||
extra_css = """
|
||||
body{font-family: Verdana,Arial,Helvetica,sans-serif; color: #333333 }
|
||||
h2{font-family: Georgia,"Times New Roman",Times,serif; color: #1B2D60}
|
||||
"""
|
||||
|
||||
conversion_options = {
|
||||
'comment' : description
|
||||
, 'tags' : category
|
||||
, 'publisher' : publisher
|
||||
, 'language' : language
|
||||
}
|
||||
|
||||
remove_tags = [
|
||||
dict(attrs={'class':['flechs','multiBox','colRecursos']})
|
||||
,dict(name=['meta','link','embed','object','iframe','base'])
|
||||
]
|
||||
keep_only_tags = [dict(attrs={'class':'Nota'})]
|
||||
remove_tags_after = dict(attrs={'id':'TextoPrint'})
|
||||
remove_tags_before = dict(attrs={'id':'FechaPrint'})
|
||||
|
||||
feeds = [
|
||||
(u'Portada' , u'http://www.eluniverso.com/rss/portada.xml' )
|
||||
,(u'Politica' , u'http://www.eluniverso.com/rss/politica.xml' )
|
||||
,(u'Economia' , u'http://www.eluniverso.com/rss/economia.xml' )
|
||||
,(u'Sucesos' , u'http://www.eluniverso.com/rss/sucesos.xml' )
|
||||
,(u'Migracion' , u'http://www.eluniverso.com/rss/migrantes_tema.xml' )
|
||||
,(u'El Pais' , u'http://www.eluniverso.com/rss/elpais.xml' )
|
||||
,(u'Internacionales' , u'http://www.eluniverso.com/rss/internacionales.xml' )
|
||||
,(u'Deportes' , u'http://www.eluniverso.com/rss/deportes.xml' )
|
||||
,(u'Gran Guayaquill' , u'http://www.eluniverso.com/rss/gran_guayaquil.xml' )
|
||||
,(u'Entretenimiento' , u'http://www.eluniverso.com/rss/arteyespectaculos.xml' )
|
||||
,(u'Vida' , u'http://www.eluniverso.com/rss/tuvida.xml' )
|
||||
,(u'Opinion' , u'http://www.eluniverso.com/rss/opinion.xml' )
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for item in soup.findAll(style=True):
|
||||
del item['style']
|
||||
return soup
|
||||
|
||||
110
resources/recipes/revista_muy.recipe
Normal file
110
resources/recipes/revista_muy.recipe
Normal file
@ -0,0 +1,110 @@
|
||||
from calibre.web.feeds.news import re
|
||||
from calibre.web.feeds.recipes import BasicNewsRecipe
|
||||
from BeautifulSoup import Tag
|
||||
|
||||
class RevistaMuyInteresante(BasicNewsRecipe):
|
||||
|
||||
title = 'Revista Muy Interesante'
|
||||
__author__ = 'Jefferson Frantz'
|
||||
description = 'Revista de divulgacion'
|
||||
timefmt = ' [%d %b, %Y]'
|
||||
language = 'es'
|
||||
|
||||
no_stylesheets = True
|
||||
remove_javascript = True
|
||||
|
||||
extra_css = ' .txt_articulo{ font-family: sans-serif; font-size: medium; text-align: justify } .contentheading{font-family: serif; font-size: large; font-weight: bold; color: #000000; text-align: center}'
|
||||
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for item in soup.findAll(style=True):
|
||||
del item['style']
|
||||
|
||||
for img_tag in soup.findAll('img'):
|
||||
imagen = img_tag
|
||||
new_tag = Tag(soup,'p')
|
||||
img_tag.replaceWith(new_tag)
|
||||
div = soup.find(attrs={'class':'article_category'})
|
||||
div.insert(0,imagen)
|
||||
break
|
||||
return soup
|
||||
|
||||
|
||||
preprocess_regexps = [
|
||||
(re.compile(r'<td class="contentheading" width="100%">.*?</td>', re.DOTALL|re.IGNORECASE), lambda match: '<td class="contentheading">' + match.group().replace('<td class="contentheading" width="100%">','').strip().replace('</td>','').strip() + '</td>'),
|
||||
|
||||
]
|
||||
|
||||
|
||||
keep_only_tags = [dict(name='div', attrs={'class':['article']}),dict(name='td', attrs={'class':['txt_articulo']})]
|
||||
|
||||
remove_tags = [
|
||||
dict(name=['object','link','script','ul'])
|
||||
,dict(name='div', attrs={'id':['comment']})
|
||||
,dict(name='td', attrs={'class':['buttonheading']})
|
||||
,dict(name='div', attrs={'class':['tags_articles']})
|
||||
,dict(name='table', attrs={'class':['pagenav']})
|
||||
]
|
||||
|
||||
remove_tags_after = dict(name='div', attrs={'class':'tags_articles'})
|
||||
|
||||
|
||||
#TO GET ARTICLES IN SECTION
|
||||
def nz_parse_section(self, url):
|
||||
soup = self.index_to_soup(url)
|
||||
div = soup.find(attrs={'class':'contenido'})
|
||||
current_articles = []
|
||||
for x in div.findAllNext(attrs={'class':['headline']}):
|
||||
a = x.find('a', href=True)
|
||||
if a is None:
|
||||
continue
|
||||
title = self.tag_to_string(a)
|
||||
url = a.get('href', False)
|
||||
if not url or not title:
|
||||
continue
|
||||
if url.startswith('/'):
|
||||
url = 'http://www.muyinteresante.es'+url
|
||||
# self.log('\t\tFound article:', title)
|
||||
# self.log('\t\t\t', url)
|
||||
current_articles.append({'title': title, 'url':url,
|
||||
'description':'', 'date':''})
|
||||
|
||||
return current_articles
|
||||
|
||||
|
||||
# To GET SECTIONS
|
||||
def parse_index(self):
|
||||
feeds = []
|
||||
for title, url in [
|
||||
('Historia',
|
||||
'http://www.muyinteresante.es/historia-articulos'),
|
||||
('Ciencia',
|
||||
'http://www.muyinteresante.es/ciencia-articulos'),
|
||||
('Naturaleza',
|
||||
'http://www.muyinteresante.es/naturaleza-articulos'),
|
||||
('Tecnología',
|
||||
'http://www.muyinteresante.es/tecnologia-articulos'),
|
||||
('Salud',
|
||||
'http://www.muyinteresante.es/salud-articulos'),
|
||||
('Más Muy',
|
||||
'http://www.muyinteresante.es/muy'),
|
||||
('Innova - Automoción',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-autos'),
|
||||
('Innova - Salud',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-salud'),
|
||||
('Innova - Medio Ambiente',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-medio-ambiente'),
|
||||
('Innova - Alimentación',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-alimentacion'),
|
||||
('Innova - Sociedad',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-sociedad'),
|
||||
('Innova - Tecnología',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-tecnologia'),
|
||||
('Innova - Ocio',
|
||||
'http://www.muyinteresante.es/articulos-innovacion-ocio'),
|
||||
]:
|
||||
articles = self.nz_parse_section(url)
|
||||
if articles:
|
||||
feeds.append((title, articles))
|
||||
return feeds
|
||||
|
||||
@ -42,7 +42,7 @@ class RMF24_opinie(BasicNewsRecipe):
|
||||
# thanks to Kovid Goyal
|
||||
def get_article_url(self, article):
|
||||
link = article.get('link')
|
||||
if 'audio' not in link:
|
||||
if '/audio,aId' not in link:
|
||||
return link
|
||||
|
||||
preprocess_regexps = [
|
||||
|
||||
78
resources/recipes/science_news_recent_issues.recipe
Normal file
78
resources/recipes/science_news_recent_issues.recipe
Normal file
@ -0,0 +1,78 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
'''
|
||||
sciencenews.org
|
||||
'''
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
class ScienceNewsIssue(BasicNewsRecipe):
|
||||
title = u'Science News Recent Issues'
|
||||
__author__ = u'Darko Miletic, Sujata Raman and Starson17'
|
||||
description = u'''Science News is an award-winning weekly
|
||||
newsmagazine covering the most important research in all fields of science.
|
||||
Its 16 pages each week are packed with short, accurate articles that appeal
|
||||
to both general readers and scientists. Published since 1922, the magazine
|
||||
now reaches about 150,000 subscribers and more than 1 million readers.
|
||||
These are the latest News Items from Science News. This recipe downloads
|
||||
the last 30 days worth of articles.'''
|
||||
category = u'Science, Technology, News'
|
||||
publisher = u'Society for Science & the Public'
|
||||
oldest_article = 30
|
||||
language = 'en'
|
||||
max_articles_per_feed = 100
|
||||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
timefmt = ' [%A, %d %B, %Y]'
|
||||
recursions = 1
|
||||
remove_attributes = ['style']
|
||||
|
||||
conversion_options = {'linearize_tables' : True
|
||||
, 'comment' : description
|
||||
, 'tags' : category
|
||||
, 'publisher' : publisher
|
||||
, 'language' : language
|
||||
}
|
||||
|
||||
extra_css = '''
|
||||
.content_description{font-family:georgia ;font-size:x-large; color:#646464 ; font-weight:bold;}
|
||||
.content_summary{font-family:georgia ;font-size:small ;color:#585858 ; font-weight:bold;}
|
||||
.content_authors{font-family:helvetica,arial ;font-size: xx-small ;color:#14487E ;}
|
||||
.content_edition{font-family:helvetica,arial ;font-size: xx-small ;}
|
||||
.exclusive{color:#FF0000 ;}
|
||||
.anonymous{color:#14487E ;}
|
||||
.content_content{font-family:helvetica,arial ;font-size: medium ; color:#000000;}
|
||||
.description{color:#585858;font-family:helvetica,arial ;font-size: large ;}
|
||||
.credit{color:#A6A6A6;font-family:helvetica,arial ;font-size: xx-small ;}
|
||||
'''
|
||||
|
||||
keep_only_tags = [ dict(name='div', attrs={'id':'column_action'}) ]
|
||||
remove_tags_after = dict(name='ul', attrs={'id':'content_functions_bottom'})
|
||||
remove_tags = [
|
||||
dict(name='ul', attrs={'id':'content_functions_bottom'})
|
||||
,dict(name='div', attrs={'id':['content_functions_top','breadcrumb_content']})
|
||||
,dict(name='img', attrs={'class':'icon'})
|
||||
,dict(name='div', attrs={'class': 'embiggen'})
|
||||
]
|
||||
|
||||
feeds = [(u"Science News Current Issues", u'http://www.sciencenews.org/view/feed/type/edition/name/issues.rss')]
|
||||
|
||||
match_regexps = [
|
||||
r'www.sciencenews.org/view/feature/id/',
|
||||
r'www.sciencenews.org/view/generic/id'
|
||||
]
|
||||
|
||||
def get_cover_url(self):
|
||||
cover_url = None
|
||||
index = 'http://www.sciencenews.org/view/home'
|
||||
soup = self.index_to_soup(index)
|
||||
link_item = soup.find(name = 'img',alt = "issue")
|
||||
if link_item:
|
||||
cover_url = 'http://www.sciencenews.org' + link_item['src'] + '.jpg'
|
||||
|
||||
return cover_url
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for tag in soup.findAll(name=['span']):
|
||||
tag.name = 'div'
|
||||
return soup
|
||||
@ -21,16 +21,20 @@ class WashingtonPost(BasicNewsRecipe):
|
||||
body{font-family:arial,helvetica,sans-serif}
|
||||
'''
|
||||
|
||||
feeds = [ ('Today\'s Highlights', 'http://www.washingtonpost.com/wp-dyn/rss/linkset/2005/03/24/LI2005032400102.xml'),
|
||||
('Politics', 'http://www.washingtonpost.com/wp-dyn/rss/politics/index.xml'),
|
||||
('Nation', 'http://www.washingtonpost.com/wp-dyn/rss/nation/index.xml'),
|
||||
('World', 'http://www.washingtonpost.com/wp-dyn/rss/world/index.xml'),
|
||||
('Business', 'http://www.washingtonpost.com/wp-dyn/rss/business/index.xml'),
|
||||
('Technology', 'http://www.washingtonpost.com/wp-dyn/rss/technology/index.xml'),
|
||||
('Health', 'http://www.washingtonpost.com/wp-dyn/rss/health/index.xml'),
|
||||
('Education', 'http://www.washingtonpost.com/wp-dyn/rss/education/index.xml'),
|
||||
('Editorials', 'http://www.washingtonpost.com/wp-dyn/rss/linkset/2005/05/30/LI2005053000331.xml'),
|
||||
]
|
||||
feeds = [ ('Today\'s Highlights', 'http://www.washingtonpost.com/wp-dyn/rss/linkset/2005/03/24/LI2005032400102.xml'),
|
||||
('Politics', 'http://www.washingtonpost.com/wp-dyn/rss/politics/index.xml'),
|
||||
('Nation', 'http://www.washingtonpost.com/wp-dyn/rss/nation/index.xml'),
|
||||
('World', 'http://www.washingtonpost.com/wp-dyn/rss/world/index.xml'),
|
||||
('Business', 'http://www.washingtonpost.com/wp-dyn/rss/business/index.xml'),
|
||||
('Technology', 'http://www.washingtonpost.com/wp-dyn/rss/technology/index.xml'),
|
||||
('Health', 'http://www.washingtonpost.com/wp-dyn/rss/health/index.xml'),
|
||||
('Education', 'http://www.washingtonpost.com/wp-dyn/rss/education/index.xml'),
|
||||
('Style',
|
||||
'http://www.washingtonpost.com/wp-dyn/rss/print/style/index.xml'),
|
||||
('Sports',
|
||||
'http://feeds.washingtonpost.com/wp-dyn/rss/linkset/2010/08/19/LI2010081904067_xml'),
|
||||
('Editorials', 'http://www.washingtonpost.com/wp-dyn/rss/linkset/2005/05/30/LI2005053000331.xml'),
|
||||
]
|
||||
|
||||
remove_tags = [{'id':['pfmnav', 'ArticleCommentsWrapper']}]
|
||||
|
||||
|
||||
@ -455,6 +455,24 @@ def prepare_string_for_xml(raw, attribute=False):
|
||||
def isbytestring(obj):
|
||||
return isinstance(obj, (str, bytes))
|
||||
|
||||
def force_unicode(obj, enc=preferred_encoding):
|
||||
if isbytestring(obj):
|
||||
try:
|
||||
obj = obj.decode(enc)
|
||||
except:
|
||||
try:
|
||||
obj = obj.decode(filesystem_encoding if enc ==
|
||||
preferred_encoding else preferred_encoding)
|
||||
except:
|
||||
try:
|
||||
obj = obj.decode('utf-8')
|
||||
except:
|
||||
obj = repr(obj)
|
||||
if isbytestring(obj):
|
||||
obj = obj.decode('utf-8')
|
||||
return obj
|
||||
|
||||
|
||||
def human_readable(size):
|
||||
""" Convert a size in bytes into a human readable form """
|
||||
divisor, suffix = 1, "B"
|
||||
|
||||
@ -2,7 +2,7 @@ __license__ = 'GPL v3'
|
||||
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
|
||||
__docformat__ = 'restructuredtext en'
|
||||
__appname__ = 'calibre'
|
||||
__version__ = '0.7.20'
|
||||
__version__ = '0.7.22'
|
||||
__author__ = "Kovid Goyal <kovid@kovidgoyal.net>"
|
||||
|
||||
import re
|
||||
|
||||
@ -460,7 +460,8 @@ from calibre.devices.hanvon.driver import N516, EB511, ALEX, AZBOOKA, THEBOOK
|
||||
from calibre.devices.edge.driver import EDGE
|
||||
from calibre.devices.teclast.driver import TECLAST_K3, NEWSMY, IPAPYRUS, SOVOS
|
||||
from calibre.devices.sne.driver import SNE
|
||||
from calibre.devices.misc import PALMPRE, AVANT, SWEEX, PDNOVEL, KOGAN, GEMEI
|
||||
from calibre.devices.misc import PALMPRE, AVANT, SWEEX, PDNOVEL, KOGAN, \
|
||||
GEMEI, VELOCITYMICRO
|
||||
from calibre.devices.folder_device.driver import FOLDER_DEVICE_FOR_CONFIG
|
||||
from calibre.devices.kobo.driver import KOBO
|
||||
|
||||
@ -572,6 +573,7 @@ plugins += [
|
||||
PDNOVEL,
|
||||
SPECTRA,
|
||||
GEMEI,
|
||||
VELOCITYMICRO,
|
||||
ITUNES,
|
||||
]
|
||||
plugins += [x for x in list(locals().values()) if isinstance(x, type) and \
|
||||
|
||||
@ -255,6 +255,9 @@ class OutputProfile(Plugin):
|
||||
#: Unsupported unicode characters to be replaced during preprocessing
|
||||
unsupported_unicode_chars = []
|
||||
|
||||
#: Number of ems that the left margin of a blockquote is rendered as
|
||||
mobi_ems_per_blockquote = 1.0
|
||||
|
||||
@classmethod
|
||||
def tags_to_string(cls, tags):
|
||||
return escape(', '.join(tags))
|
||||
@ -564,6 +567,7 @@ class KindleOutput(OutputProfile):
|
||||
supports_mobi_indexing = True
|
||||
periodical_date_in_title = False
|
||||
ratings_char = u'\u2605'
|
||||
mobi_ems_per_blockquote = 2.0
|
||||
|
||||
@classmethod
|
||||
def tags_to_string(cls, tags):
|
||||
@ -582,6 +586,7 @@ class KindleDXOutput(OutputProfile):
|
||||
comic_screen_size = (741, 1022)
|
||||
supports_mobi_indexing = True
|
||||
periodical_date_in_title = False
|
||||
mobi_ems_per_blockquote = 2.0
|
||||
|
||||
@classmethod
|
||||
def tags_to_string(cls, tags):
|
||||
|
||||
@ -16,7 +16,8 @@ class FOLDER_DEVICE_FOR_CONFIG(USBMS):
|
||||
description = _('Use an arbitrary folder as a device.')
|
||||
author = 'John Schember/Charles Haley'
|
||||
supported_platforms = ['windows', 'osx', 'linux']
|
||||
FORMATS = ['epub', 'fb2', 'mobi', 'azw', 'lrf', 'tcr', 'pmlz', 'lit', 'rtf', 'rb', 'pdf', 'oeb', 'txt', 'pdb']
|
||||
FORMATS = ['epub', 'fb2', 'mobi', 'azw', 'lrf', 'tcr', 'pmlz', 'lit',
|
||||
'rtf', 'rb', 'pdf', 'oeb', 'txt', 'pdb', 'prc']
|
||||
VENDOR_ID = 0xffff
|
||||
PRODUCT_ID = 0xffff
|
||||
BCD = 0xffff
|
||||
@ -30,7 +31,7 @@ class FOLDER_DEVICE(USBMS):
|
||||
description = _('Use an arbitrary folder as a device.')
|
||||
author = 'John Schember/Charles Haley'
|
||||
supported_platforms = ['windows', 'osx', 'linux']
|
||||
FORMATS = ['epub', 'fb2', 'mobi', 'azw', 'lrf', 'tcr', 'pmlz', 'lit', 'rtf', 'rb', 'pdf', 'oeb', 'txt', 'pdb']
|
||||
FORMATS = FOLDER_DEVICE_FOR_CONFIG.FORMATS
|
||||
|
||||
VENDOR_ID = 0xffff
|
||||
PRODUCT_ID = 0xffff
|
||||
|
||||
@ -7,7 +7,7 @@ __docformat__ = 'restructuredtext en'
|
||||
'''
|
||||
Device driver for Hanvon devices
|
||||
'''
|
||||
import re
|
||||
import re, os
|
||||
|
||||
from calibre.devices.usbms.driver import USBMS
|
||||
|
||||
@ -59,18 +59,59 @@ class ALEX(N516):
|
||||
description = _('Communicate with the SpringDesign Alex eBook reader.')
|
||||
author = 'Kovid Goyal'
|
||||
|
||||
FORMATS = ['epub', 'pdf']
|
||||
FORMATS = ['epub', 'fb2', 'pdf']
|
||||
VENDOR_NAME = 'ALEX'
|
||||
WINDOWS_MAIN_MEM = 'READER'
|
||||
|
||||
MAIN_MEMORY_VOLUME_LABEL = 'Alex Internal Memory'
|
||||
|
||||
EBOOK_DIR_MAIN = 'eBooks'
|
||||
SUPPORTS_SUB_DIRS = True
|
||||
SUPPORTS_SUB_DIRS = False
|
||||
THUMBNAIL_HEIGHT = 120
|
||||
|
||||
def can_handle(self, device_info, debug=False):
|
||||
return is_alex(device_info)
|
||||
|
||||
def alex_cpath(self, file_abspath):
|
||||
base = os.path.dirname(file_abspath)
|
||||
name = os.path.splitext(os.path.basename(file_abspath))[0] + '.png'
|
||||
return os.path.join(base, 'covers', name)
|
||||
|
||||
def upload_cover(self, path, filename, metadata):
|
||||
from calibre.ebooks import calibre_cover
|
||||
from calibre.utils.magick.draw import thumbnail
|
||||
coverdata = getattr(metadata, 'thumbnail', None)
|
||||
if coverdata and coverdata[2]:
|
||||
cover = coverdata[2]
|
||||
else:
|
||||
cover = calibre_cover(metadata.get('title', _('Unknown')),
|
||||
metadata.get('authors', _('Unknown')))
|
||||
|
||||
cover = thumbnail(cover, width=self.THUMBNAIL_HEIGHT,
|
||||
height=self.THUMBNAIL_HEIGHT, fmt='png')[-1]
|
||||
|
||||
cpath = self.alex_cpath(os.path.join(path, filename))
|
||||
cdir = os.path.dirname(cpath)
|
||||
if not os.path.exists(cdir):
|
||||
os.makedirs(cdir)
|
||||
with open(cpath, 'wb') as coverfile:
|
||||
coverfile.write(cover)
|
||||
|
||||
def delete_books(self, paths, end_session=True):
|
||||
for i, path in enumerate(paths):
|
||||
self.report_progress((i+1) / float(len(paths)), _('Removing books from device...'))
|
||||
path = self.normalize_path(path)
|
||||
if os.path.exists(path):
|
||||
# Delete the ebook
|
||||
os.unlink(path)
|
||||
try:
|
||||
cpath = self.alex_cpath(path)
|
||||
if os.path.exists(cpath):
|
||||
os.remove(cpath)
|
||||
except:
|
||||
pass
|
||||
self.report_progress(1.0, _('Removing books from device...'))
|
||||
|
||||
class AZBOOKA(ALEX):
|
||||
|
||||
name = 'Azbooka driver'
|
||||
@ -83,10 +124,13 @@ class AZBOOKA(ALEX):
|
||||
MAIN_MEMORY_VOLUME_LABEL = 'Azbooka Internal Memory'
|
||||
|
||||
EBOOK_DIR_MAIN = ''
|
||||
SUPPORTS_SUB_DIRS = True
|
||||
|
||||
def can_handle(self, device_info, debug=False):
|
||||
return not is_alex(device_info)
|
||||
|
||||
def upload_cover(self, path, filename, metadata):
|
||||
pass
|
||||
|
||||
class EB511(USBMS):
|
||||
name = 'Elonex EB 511 driver'
|
||||
|
||||
@ -20,7 +20,7 @@ class IREXDR1000(USBMS):
|
||||
|
||||
# Ordered list of supported formats
|
||||
# Be sure these have an entry in calibre.devices.mime
|
||||
FORMATS = ['epub', 'mobi', 'prc', 'html', 'pdf', 'txt']
|
||||
FORMATS = ['epub', 'mobi', 'prc', 'html', 'pdf', 'djvu', 'txt']
|
||||
|
||||
VENDOR_ID = [0x1e6b]
|
||||
PRODUCT_ID = [0x001]
|
||||
|
||||
@ -108,6 +108,24 @@ class PDNOVEL(USBMS):
|
||||
with open('%s.jpg' % os.path.join(path, filename), 'wb') as coverfile:
|
||||
coverfile.write(coverdata[2])
|
||||
|
||||
class VELOCITYMICRO(USBMS):
|
||||
name = 'VelocityMicro device interface'
|
||||
gui_name = 'VelocityMicro'
|
||||
description = _('Communicate with the VelocityMicro')
|
||||
author = 'Kovid Goyal'
|
||||
supported_platforms = ['windows', 'linux', 'osx']
|
||||
FORMATS = ['epub', 'pdb', 'txt', 'html', 'pdf']
|
||||
|
||||
VENDOR_ID = [0x18d1]
|
||||
PRODUCT_ID = [0xb015]
|
||||
BCD = [0x224]
|
||||
|
||||
VENDOR_NAME = 'ANDROID'
|
||||
WINDOWS_MAIN_MEM = WINDOWS_CARD_A_MEM = '__UMS_COMPOSITE'
|
||||
|
||||
EBOOK_DIR_MAIN = 'eBooks'
|
||||
SUPPORTS_SUB_DIRS = False
|
||||
|
||||
class GEMEI(USBMS):
|
||||
name = 'Gemei Device Interface'
|
||||
gui_name = 'GM2000'
|
||||
|
||||
@ -63,6 +63,8 @@ class PRS505(USBMS):
|
||||
'series, tags, authors'
|
||||
EXTRA_CUSTOMIZATION_DEFAULT = ', '.join(['series', 'tags'])
|
||||
|
||||
plugboard = None
|
||||
|
||||
def windows_filter_pnp_id(self, pnp_id):
|
||||
return '_LAUNCHER' in pnp_id
|
||||
|
||||
@ -150,7 +152,7 @@ class PRS505(USBMS):
|
||||
else:
|
||||
collections = []
|
||||
debug_print('PRS505: collection fields:', collections)
|
||||
c.update(blists, collections)
|
||||
c.update(blists, collections, self.plugboard)
|
||||
c.write()
|
||||
|
||||
USBMS.sync_booklists(self, booklists, end_session=end_session)
|
||||
@ -163,3 +165,9 @@ class PRS505(USBMS):
|
||||
c.write()
|
||||
debug_print('PRS505: finished rebuild_collections')
|
||||
|
||||
def use_plugboard_ext(self):
|
||||
return 'device_db'
|
||||
|
||||
def set_plugboard(self, pb):
|
||||
debug_print('PRS505: use plugboard', pb)
|
||||
self.plugboard = pb
|
||||
|
||||
@ -325,12 +325,6 @@ class XMLCache(object):
|
||||
for book in bl:
|
||||
record = lpath_map.get(book.lpath, None)
|
||||
if record is not None:
|
||||
title = record.get('title', None)
|
||||
if title is not None and title != book.title:
|
||||
debug_print('Renaming title', book.title, 'to', title)
|
||||
book.title = title
|
||||
# Don't set the author, because the reader strips all but
|
||||
# the first author.
|
||||
for thumbnail in record.xpath(
|
||||
'descendant::*[local-name()="thumbnail"]'):
|
||||
for img in thumbnail.xpath(
|
||||
@ -350,7 +344,7 @@ class XMLCache(object):
|
||||
# }}}
|
||||
|
||||
# Update XML from JSON {{{
|
||||
def update(self, booklists, collections_attributes):
|
||||
def update(self, booklists, collections_attributes, plugboard):
|
||||
debug_print('Starting update', collections_attributes)
|
||||
use_tz_var = False
|
||||
for i, booklist in booklists.items():
|
||||
@ -365,8 +359,13 @@ class XMLCache(object):
|
||||
record = lpath_map.get(book.lpath, None)
|
||||
if record is None:
|
||||
record = self.create_text_record(root, i, book.lpath)
|
||||
if plugboard is not None:
|
||||
newmi = book.deepcopy()
|
||||
newmi.template_to_attribute(book, plugboard)
|
||||
else:
|
||||
newmi = book
|
||||
(gtz_count, ltz_count, use_tz_var) = \
|
||||
self.update_text_record(record, book, path, i,
|
||||
self.update_text_record(record, newmi, path, i,
|
||||
gtz_count, ltz_count, use_tz_var)
|
||||
# Ensure the collections in the XML database are recorded for
|
||||
# this book
|
||||
|
||||
@ -151,7 +151,8 @@ class CHMReader(CHMFile):
|
||||
continue
|
||||
raise
|
||||
self._extracted = True
|
||||
files = os.listdir(output_dir)
|
||||
files = [x for x in os.listdir(output_dir) if
|
||||
os.path.isfile(os.path.join(output_dir, x))]
|
||||
if self.hhc_path not in files:
|
||||
for f in files:
|
||||
if f.lower() == self.hhc_path.lower():
|
||||
|
||||
@ -707,7 +707,7 @@ OptionRecommendation(name='timestamp',
|
||||
if mi.cover.startswith('http:') or mi.cover.startswith('https:'):
|
||||
mi.cover = self.download_cover(mi.cover)
|
||||
ext = mi.cover.rpartition('.')[-1].lower().strip()
|
||||
if ext not in ('png', 'jpg', 'jpeg'):
|
||||
if ext not in ('png', 'jpg', 'jpeg', 'gif'):
|
||||
ext = 'jpg'
|
||||
mi.cover_data = (ext, open(mi.cover, 'rb').read())
|
||||
mi.cover = None
|
||||
|
||||
@ -44,7 +44,15 @@ def author_to_author_sort(author):
|
||||
def authors_to_sort_string(authors):
|
||||
return ' & '.join(map(author_to_author_sort, authors))
|
||||
|
||||
_title_pat = re.compile('^(A|The|An)\s+', re.IGNORECASE)
|
||||
try:
|
||||
_title_pat = re.compile(tweaks.get('title_sort_articles',
|
||||
r'^(A|The|An)\s+'), re.IGNORECASE)
|
||||
except:
|
||||
print 'Error in title sort pattern'
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
_title_pat = re.compile('^(A|The|An)\s+', re.IGNORECASE)
|
||||
|
||||
_ignore_starts = u'\'"'+u''.join(unichr(x) for x in range(0x2018, 0x201e)+[0x2032, 0x2033])
|
||||
|
||||
def title_sort(title):
|
||||
|
||||
@ -114,7 +114,8 @@ SC_COPYABLE_FIELDS = SOCIAL_METADATA_FIELDS.union(
|
||||
PUBLICATION_METADATA_FIELDS).union(
|
||||
BOOK_STRUCTURE_FIELDS).union(
|
||||
DEVICE_METADATA_FIELDS).union(
|
||||
CALIBRE_METADATA_FIELDS) - \
|
||||
CALIBRE_METADATA_FIELDS).union(
|
||||
TOP_LEVEL_CLASSIFIERS) - \
|
||||
SC_FIELDS_NOT_COPIED.union(
|
||||
SC_FIELDS_COPY_NOT_NULL)
|
||||
|
||||
|
||||
@ -8,6 +8,7 @@ __docformat__ = 'restructuredtext en'
|
||||
import copy, traceback
|
||||
|
||||
from calibre import prints
|
||||
from calibre.constants import DEBUG
|
||||
from calibre.ebooks.metadata.book import SC_COPYABLE_FIELDS
|
||||
from calibre.ebooks.metadata.book import SC_FIELDS_COPY_NOT_NULL
|
||||
from calibre.ebooks.metadata.book import STANDARD_METADATA_FIELDS
|
||||
@ -50,6 +51,8 @@ class SafeFormat(TemplateFormatter):
|
||||
return ''
|
||||
return v
|
||||
except:
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
return key
|
||||
|
||||
composite_formatter = SafeFormat()
|
||||
@ -320,8 +323,8 @@ class Metadata(object):
|
||||
else:
|
||||
self.set(dest, val)
|
||||
except:
|
||||
traceback.print_exc()
|
||||
pass
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
|
||||
# Old Metadata API {{{
|
||||
def print_all_attributes(self):
|
||||
|
||||
@ -33,7 +33,10 @@ def get_metadata(stream):
|
||||
le = XPath('descendant::fb2:last-name')(au)
|
||||
if le:
|
||||
lname = tostring(le[0])
|
||||
author += ' '+lname
|
||||
if author:
|
||||
author += ' '+lname
|
||||
else:
|
||||
author = lname
|
||||
if author:
|
||||
authors.append(author)
|
||||
if len(authors) == 1 and author is not None:
|
||||
|
||||
@ -108,7 +108,8 @@ def _get_metadata(stream, stream_type, use_libprs_metadata,
|
||||
base = metadata_from_filename(name, pat=pattern)
|
||||
if force_read_metadata or is_recipe(name) or prefs['read_file_metadata']:
|
||||
mi = get_file_type_metadata(stream, stream_type)
|
||||
if base.title == os.path.splitext(name)[0] and base.authors is None:
|
||||
if base.title == os.path.splitext(name)[0] and \
|
||||
base.is_null('authors') and base.is_null('isbn'):
|
||||
# Assume that there was no metadata in the file and the user set pattern
|
||||
# to match meta info from the file name did not match.
|
||||
# The regex is meant to match the standard format filenames are written
|
||||
|
||||
@ -184,13 +184,14 @@ class MobiMLizer(object):
|
||||
elif tag in NESTABLE_TAGS and istate.rendered:
|
||||
para = wrapper = bstate.nested[-1]
|
||||
elif left > 0 and indent >= 0:
|
||||
para = wrapper = etree.SubElement(parent, XHTML('div'))
|
||||
ems = self.profile.mobi_ems_per_blockquote
|
||||
para = wrapper = etree.SubElement(parent, XHTML('blockquote'))
|
||||
para = wrapper
|
||||
emleft = int(round(left / self.profile.fbase)) - 1
|
||||
emleft = int(round(left / self.profile.fbase)) - ems
|
||||
emleft = min((emleft, 10))
|
||||
while emleft > 0:
|
||||
para = etree.SubElement(para, XHTML('div'))
|
||||
emleft -= 1
|
||||
para = etree.SubElement(para, XHTML('blockquote'))
|
||||
emleft -= ems
|
||||
else:
|
||||
para = wrapper = etree.SubElement(parent, XHTML('p'))
|
||||
bstate.inline = bstate.para = para
|
||||
@ -437,4 +438,4 @@ class MobiMLizer(object):
|
||||
bstate.vpadding += vpadding
|
||||
if bstate.nested and bstate.nested[-1].tag == elem.tag:
|
||||
bstate.nested.pop()
|
||||
istates.pop()
|
||||
istates.pop()
|
||||
@ -234,7 +234,7 @@ class MobiReader(object):
|
||||
self.debug = debug
|
||||
self.embedded_mi = None
|
||||
self.base_css_rules = textwrap.dedent('''
|
||||
blockquote { margin: 0em 0em 0em 1em; text-align: justify }
|
||||
blockquote { margin: 0em 0em 0em 2em; text-align: justify }
|
||||
|
||||
p { margin: 0em; text-align: justify }
|
||||
|
||||
|
||||
@ -15,7 +15,6 @@ from struct import pack
|
||||
import time
|
||||
from urlparse import urldefrag
|
||||
|
||||
from PIL import Image
|
||||
from cStringIO import StringIO
|
||||
from calibre.ebooks.mobi.langcodes import iana2mobi
|
||||
from calibre.ebooks.mobi.mobiml import MBP_NS
|
||||
@ -28,6 +27,7 @@ from calibre.ebooks.oeb.base import namespace
|
||||
from calibre.ebooks.oeb.base import prefixname
|
||||
from calibre.ebooks.oeb.base import urlnormalize
|
||||
from calibre.ebooks.compression.palmdoc import compress_doc
|
||||
from calibre.utils.magick.draw import Image, save_cover_data_to, thumbnail
|
||||
|
||||
INDEXING = True
|
||||
FCIS_FLIS = True
|
||||
@ -111,46 +111,18 @@ def align_block(raw, multiple=4, pad='\0'):
|
||||
return raw + pad*(multiple - extra)
|
||||
|
||||
def rescale_image(data, maxsizeb, dimen=None):
|
||||
image = Image.open(StringIO(data))
|
||||
format = image.format
|
||||
changed = False
|
||||
if image.format not in ('JPEG', 'GIF'):
|
||||
width, height = image.size
|
||||
area = width * height
|
||||
if area <= 40000:
|
||||
format = 'GIF'
|
||||
else:
|
||||
image = image.convert('RGBA')
|
||||
format = 'JPEG'
|
||||
changed = True
|
||||
if dimen is not None:
|
||||
image.thumbnail(dimen, Image.ANTIALIAS)
|
||||
changed = True
|
||||
if changed:
|
||||
data = StringIO()
|
||||
image.save(data, format)
|
||||
data = data.getvalue()
|
||||
if len(data) <= maxsizeb:
|
||||
return data
|
||||
image = image.convert('RGBA')
|
||||
for quality in xrange(95, -1, -1):
|
||||
data = StringIO()
|
||||
image.save(data, 'JPEG', quality=quality)
|
||||
data = data.getvalue()
|
||||
if len(data) <= maxsizeb:
|
||||
return data
|
||||
width, height = image.size
|
||||
for scale in xrange(99, 0, -1):
|
||||
scale = scale / 100.
|
||||
data = StringIO()
|
||||
scaled = image.copy()
|
||||
size = (int(width * scale), (height * scale))
|
||||
scaled.thumbnail(size, Image.ANTIALIAS)
|
||||
scaled.save(data, 'JPEG', quality=0)
|
||||
data = data.getvalue()
|
||||
if len(data) <= maxsizeb:
|
||||
return data
|
||||
# Well, we tried?
|
||||
return thumbnail(data, width=dimen, height=dimen)[-1]
|
||||
# Replace transparent pixels with white pixels and convert to JPEG
|
||||
data = save_cover_data_to(data, 'img.jpg', return_data=True)
|
||||
scale = 0.9
|
||||
while len(data) >= maxsizeb and scale >= 0.05:
|
||||
img = Image()
|
||||
img.load(data)
|
||||
w, h = img.size
|
||||
img.size = (int(scale*w), int(scale*h))
|
||||
data = img.export('jpg')
|
||||
scale -= 0.05
|
||||
return data
|
||||
|
||||
class Serializer(object):
|
||||
|
||||
@ -654,8 +654,6 @@ class Metadata(object):
|
||||
if predicate(x):
|
||||
l.remove(x)
|
||||
|
||||
|
||||
|
||||
def __getitem__(self, key):
|
||||
return self.items[key]
|
||||
|
||||
|
||||
@ -132,17 +132,23 @@ class OEBReader(object):
|
||||
if not mi.language:
|
||||
mi.language = get_lang().replace('_', '-')
|
||||
self.oeb.metadata.add('language', mi.language)
|
||||
if not mi.title:
|
||||
mi.title = self.oeb.translate(__('Unknown'))
|
||||
if not mi.authors:
|
||||
mi.authors = [self.oeb.translate(__('Unknown'))]
|
||||
if not mi.book_producer:
|
||||
mi.book_producer = '%(a)s (%(v)s) [http://%(a)s.kovidgoyal.net]'%\
|
||||
mi.book_producer = '%(a)s (%(v)s) [http://%(a)s-ebook.com]'%\
|
||||
dict(a=__appname__, v=__version__)
|
||||
meta_info_to_oeb_metadata(mi, self.oeb.metadata, self.logger)
|
||||
self.oeb.metadata.add('identifier', str(uuid.uuid4()), id='uuid_id',
|
||||
scheme='uuid')
|
||||
m = self.oeb.metadata
|
||||
m.add('identifier', str(uuid.uuid4()), id='uuid_id', scheme='uuid')
|
||||
self.oeb.uid = self.oeb.metadata.identifier[-1]
|
||||
if not m.title:
|
||||
m.add('title', self.oeb.translate(__('Unknown')))
|
||||
has_aut = False
|
||||
for x in m.creator:
|
||||
if getattr(x, 'role', '').lower() in ('', 'aut'):
|
||||
has_aut = True
|
||||
break
|
||||
if not has_aut:
|
||||
m.add('creator', self.oeb.translate(__('Unknown')), role='aut')
|
||||
|
||||
|
||||
def _manifest_prune_invalid(self):
|
||||
'''
|
||||
|
||||
@ -31,12 +31,14 @@ class CoverManager(object):
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<svg version="1.1" xmlns="http://www.w3.org/2000/svg"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
width="100%%" height="100%%" viewBox="__viewbox__"
|
||||
preserveAspectRatio="__ar__">
|
||||
<image width="__width__" height="__height__" xlink:href="%s"/>
|
||||
</svg>
|
||||
<div>
|
||||
<svg version="1.1" xmlns="http://www.w3.org/2000/svg"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
width="100%%" height="100%%" viewBox="__viewbox__"
|
||||
preserveAspectRatio="__ar__">
|
||||
<image width="__width__" height="__height__" xlink:href="%s"/>
|
||||
</svg>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
''')
|
||||
|
||||
@ -39,7 +39,7 @@ class RescaleImages(object):
|
||||
if item.media_type.startswith('image'):
|
||||
ext = item.media_type.split('/')[-1].upper()
|
||||
if ext == 'JPG': ext = 'JPEG'
|
||||
if ext not in ('PNG', 'JPEG'):
|
||||
if ext not in ('PNG', 'JPEG', 'GIF'):
|
||||
ext = 'JPEG'
|
||||
|
||||
raw = item.data
|
||||
|
||||
@ -3,6 +3,7 @@ __copyright__ = '2008, Kovid Goyal <kovid at kovidgoyal.net>'
|
||||
""" The GUI """
|
||||
import os, sys, Queue, threading
|
||||
from threading import RLock
|
||||
from urllib import unquote
|
||||
|
||||
from PyQt4.Qt import QVariant, QFileInfo, QObject, SIGNAL, QBuffer, Qt, \
|
||||
QByteArray, QTranslator, QCoreApplication, QThread, \
|
||||
@ -505,6 +506,11 @@ class FileDialog(QObject):
|
||||
fs = QFileDialog.getOpenFileNames(parent, title, initial_dir, ftext, "")
|
||||
for f in fs:
|
||||
f = unicode(f)
|
||||
if not f: continue
|
||||
if not os.path.exists(f):
|
||||
# QFileDialog for some reason quotes spaces
|
||||
# on linux if there is more than one space in a row
|
||||
f = unquote(f)
|
||||
if f and os.path.exists(f):
|
||||
self.selected_files.append(f)
|
||||
else:
|
||||
|
||||
@ -234,13 +234,14 @@ class AddAction(InterfaceAction):
|
||||
self.gui.set_books_in_library(booklists=[model.db], reset=True)
|
||||
self.gui.refresh_ondevice()
|
||||
|
||||
def add_books_from_device(self, view):
|
||||
rows = view.selectionModel().selectedRows()
|
||||
if not rows or len(rows) == 0:
|
||||
d = error_dialog(self.gui, _('Add to library'), _('No book selected'))
|
||||
d.exec_()
|
||||
return
|
||||
paths = [p for p in view._model.paths(rows) if p is not None]
|
||||
def add_books_from_device(self, view, paths=None):
|
||||
if paths is None:
|
||||
rows = view.selectionModel().selectedRows()
|
||||
if not rows or len(rows) == 0:
|
||||
d = error_dialog(self.gui, _('Add to library'), _('No book selected'))
|
||||
d.exec_()
|
||||
return
|
||||
paths = [p for p in view.model().paths(rows) if p is not None]
|
||||
ve = self.gui.device_manager.device.VIRTUAL_BOOK_EXTENSIONS
|
||||
def ext(x):
|
||||
ans = os.path.splitext(x)[1]
|
||||
@ -261,7 +262,7 @@ class AddAction(InterfaceAction):
|
||||
return
|
||||
from calibre.gui2.add import Adder
|
||||
self.__adder_func = partial(self._add_from_device_adder, on_card=None,
|
||||
model=view._model)
|
||||
model=view.model())
|
||||
self._adder = Adder(self.gui, self.gui.library_view.model().db,
|
||||
self.Dispatcher(self.__adder_func), spare_server=self.gui.spare_server)
|
||||
self._adder.add(paths)
|
||||
|
||||
@ -181,5 +181,6 @@ class ConvertAction(InterfaceAction):
|
||||
self.gui.tags_view.recount()
|
||||
if self.gui.current_view() is self.gui.library_view:
|
||||
current = self.gui.library_view.currentIndex()
|
||||
self.gui.library_view.model().current_changed(current, QModelIndex())
|
||||
if current.isValid():
|
||||
self.gui.library_view.model().current_changed(current, QModelIndex())
|
||||
|
||||
|
||||
@ -21,7 +21,10 @@ from calibre.gui2.convert import Widget
|
||||
def create_opf_file(db, book_id):
|
||||
mi = db.get_metadata(book_id, index_is_id=True)
|
||||
mi.application_id = uuid.uuid4()
|
||||
old_cover = mi.cover
|
||||
mi.cover = None
|
||||
raw = metadata_to_opf(mi)
|
||||
mi.cover = old_cover
|
||||
opf_file = PersistentTemporaryFile('.opf')
|
||||
opf_file.write(raw)
|
||||
opf_file.close()
|
||||
|
||||
@ -23,7 +23,7 @@ from calibre.gui2 import config, error_dialog, Dispatcher, dynamic, \
|
||||
warning_dialog, \
|
||||
question_dialog, info_dialog, choose_dir
|
||||
from calibre.ebooks.metadata import authors_to_string
|
||||
from calibre import preferred_encoding, prints
|
||||
from calibre import preferred_encoding, prints, force_unicode
|
||||
from calibre.utils.filenames import ascii_filename
|
||||
from calibre.devices.errors import FreeSpaceError
|
||||
from calibre.utils.smtp import compose_mail, sendmail, extract_email_address, \
|
||||
@ -310,7 +310,13 @@ class DeviceManager(Thread): # {{{
|
||||
self.device.sync_booklists(booklists, end_session=False)
|
||||
return self.device.card_prefix(end_session=False), self.device.free_space()
|
||||
|
||||
def sync_booklists(self, done, booklists):
|
||||
def sync_booklists(self, done, booklists, plugboards):
|
||||
if hasattr(self.connected_device, 'use_plugboard_ext') and \
|
||||
callable(self.connected_device.use_plugboard_ext):
|
||||
ext = self.connected_device.use_plugboard_ext()
|
||||
if ext is not None:
|
||||
self.connected_device.set_plugboard(
|
||||
self.find_plugboard(ext, plugboards))
|
||||
return self.create_job(self._sync_booklists, done, args=[booklists],
|
||||
description=_('Send metadata to device'))
|
||||
|
||||
@ -319,28 +325,31 @@ class DeviceManager(Thread): # {{{
|
||||
args=[booklist, on_card],
|
||||
description=_('Send collections to device'))
|
||||
|
||||
def find_plugboard(self, ext, plugboards):
|
||||
dev_name = self.connected_device.__class__.__name__
|
||||
cpb = None
|
||||
if ext in plugboards:
|
||||
cpb = plugboards[ext]
|
||||
elif plugboard_any_format_value in plugboards:
|
||||
cpb = plugboards[plugboard_any_format_value]
|
||||
if cpb is not None:
|
||||
if dev_name in cpb:
|
||||
cpb = cpb[dev_name]
|
||||
elif plugboard_any_device_value in cpb:
|
||||
cpb = cpb[plugboard_any_device_value]
|
||||
else:
|
||||
cpb = None
|
||||
if DEBUG:
|
||||
prints('Device using plugboard', ext, dev_name, cpb)
|
||||
return cpb
|
||||
|
||||
def _upload_books(self, files, names, on_card=None, metadata=None, plugboards=None):
|
||||
'''Upload books to device: '''
|
||||
if metadata and files and len(metadata) == len(files):
|
||||
for f, mi in zip(files, metadata):
|
||||
if isinstance(f, unicode):
|
||||
ext = f.rpartition('.')[-1].lower()
|
||||
dev_name = self.connected_device.__class__.__name__
|
||||
cpb = None
|
||||
if ext in plugboards:
|
||||
cpb = plugboards[ext]
|
||||
elif plugboard_any_format_value in plugboards:
|
||||
cpb = plugboards[plugboard_any_format_value]
|
||||
if cpb is not None:
|
||||
if dev_name in cpb:
|
||||
cpb = cpb[dev_name]
|
||||
elif plugboard_any_device_value in cpb:
|
||||
cpb = cpb[plugboard_any_device_value]
|
||||
else:
|
||||
cpb = None
|
||||
|
||||
if DEBUG:
|
||||
prints('Using plugboard', ext, dev_name, cpb)
|
||||
cpb = self.find_plugboard(ext, plugboards)
|
||||
if ext:
|
||||
try:
|
||||
if DEBUG:
|
||||
@ -964,12 +973,12 @@ class DeviceMixin(object): # {{{
|
||||
for jobname, exception, tb in results:
|
||||
title = jobname.partition(':')[-1]
|
||||
if exception is not None:
|
||||
errors.append([title, exception, tb])
|
||||
errors.append(list(map(force_unicode, [title, exception, tb])))
|
||||
else:
|
||||
good.append(title)
|
||||
if errors:
|
||||
errors = '\n'.join([
|
||||
'%s\n\n%s\n%s\n' %
|
||||
errors = u'\n'.join([
|
||||
u'%s\n\n%s\n%s\n' %
|
||||
(title, e, tb) for \
|
||||
title, e, tb in errors
|
||||
])
|
||||
@ -1247,8 +1256,9 @@ class DeviceMixin(object): # {{{
|
||||
'''
|
||||
Upload metadata to device.
|
||||
'''
|
||||
plugboards = self.library_view.model().db.prefs.get('plugboards', {})
|
||||
self.device_manager.sync_booklists(Dispatcher(self.metadata_synced),
|
||||
self.booklists())
|
||||
self.booklists(), plugboards)
|
||||
|
||||
def metadata_synced(self, job):
|
||||
'''
|
||||
@ -1502,8 +1512,10 @@ class DeviceMixin(object): # {{{
|
||||
|
||||
if update_metadata:
|
||||
if self.device_manager.is_device_connected:
|
||||
plugboards = self.library_view.model().db.prefs.get('plugboards', {})
|
||||
self.device_manager.sync_booklists(
|
||||
Dispatcher(self.metadata_synced), booklists)
|
||||
Dispatcher(self.metadata_synced), booklists,
|
||||
plugboards)
|
||||
return update_metadata
|
||||
# }}}
|
||||
|
||||
|
||||
122
src/calibre/gui2/dialogs/check_library.py
Normal file
122
src/calibre/gui2/dialogs/check_library.py
Normal file
@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env python
|
||||
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
|
||||
__docformat__ = 'restructuredtext en'
|
||||
__license__ = 'GPL v3'
|
||||
|
||||
from PyQt4.Qt import QDialog, QVBoxLayout, QHBoxLayout, QTreeWidget, QLabel, \
|
||||
QPushButton, QDialogButtonBox, QApplication, QTreeWidgetItem, \
|
||||
QLineEdit
|
||||
|
||||
from calibre.library.check_library import CheckLibrary, CHECKS
|
||||
|
||||
class Item(QTreeWidgetItem):
|
||||
pass
|
||||
|
||||
class CheckLibraryDialog(QDialog):
|
||||
|
||||
def __init__(self, parent, db):
|
||||
QDialog.__init__(self, parent)
|
||||
self.db = db
|
||||
|
||||
self.setWindowTitle(_('Check Library'))
|
||||
|
||||
self._layout = QVBoxLayout(self)
|
||||
self.setLayout(self._layout)
|
||||
|
||||
self.log = QTreeWidget(self)
|
||||
self._layout.addWidget(self.log)
|
||||
|
||||
self.check = QPushButton(_('Run the check'))
|
||||
self.check.setDefault(False)
|
||||
self.check.clicked.connect(self.run_the_check)
|
||||
self.copy = QPushButton(_('Copy to clipboard'))
|
||||
self.copy.setDefault(False)
|
||||
self.copy.clicked.connect(self.copy_to_clipboard)
|
||||
self.ok = QPushButton('&OK')
|
||||
self.ok.setDefault(True)
|
||||
self.ok.clicked.connect(self.accept)
|
||||
self.cancel = QPushButton('&Cancel')
|
||||
self.cancel.setDefault(False)
|
||||
self.cancel.clicked.connect(self.reject)
|
||||
self.bbox = QDialogButtonBox(self)
|
||||
self.bbox.addButton(self.copy, QDialogButtonBox.ActionRole)
|
||||
self.bbox.addButton(self.check, QDialogButtonBox.ActionRole)
|
||||
self.bbox.addButton(self.cancel, QDialogButtonBox.RejectRole)
|
||||
self.bbox.addButton(self.ok, QDialogButtonBox.AcceptRole)
|
||||
|
||||
h = QHBoxLayout()
|
||||
ln = QLabel(_('Names to ignore:'))
|
||||
h.addWidget(ln)
|
||||
self.name_ignores = QLineEdit()
|
||||
self.name_ignores.setText(db.prefs.get('check_library_ignore_names', ''))
|
||||
ln.setBuddy(self.name_ignores)
|
||||
h.addWidget(self.name_ignores)
|
||||
le = QLabel(_('Extensions to ignore'))
|
||||
h.addWidget(le)
|
||||
self.ext_ignores = QLineEdit()
|
||||
self.ext_ignores.setText(db.prefs.get('check_library_ignore_extensions', ''))
|
||||
le.setBuddy(self.ext_ignores)
|
||||
h.addWidget(self.ext_ignores)
|
||||
self._layout.addLayout(h)
|
||||
|
||||
self._layout.addWidget(self.bbox)
|
||||
self.resize(750, 500)
|
||||
self.bbox.setEnabled(True)
|
||||
|
||||
self.run_the_check()
|
||||
|
||||
def accept(self):
|
||||
self.db.prefs['check_library_ignore_extensions'] = \
|
||||
unicode(self.ext_ignores.text())
|
||||
self.db.prefs['check_library_ignore_names'] = \
|
||||
unicode(self.name_ignores.text())
|
||||
QDialog.accept(self)
|
||||
|
||||
def box_to_list(self, txt):
|
||||
return [f.strip() for f in txt.split(',') if f.strip()]
|
||||
|
||||
def run_the_check(self):
|
||||
checker = CheckLibrary(self.db.library_path, self.db)
|
||||
checker.scan_library(self.box_to_list(unicode(self.name_ignores.text())),
|
||||
self.box_to_list(unicode(self.ext_ignores.text())))
|
||||
|
||||
plaintext = []
|
||||
|
||||
def builder(tree, checker, check):
|
||||
attr = check[0]
|
||||
list = getattr(checker, attr, None)
|
||||
if list is None:
|
||||
return
|
||||
|
||||
h = check[1]
|
||||
tl = Item([h])
|
||||
for problem in list:
|
||||
it = Item()
|
||||
it.setText(0, problem[0])
|
||||
it.setText(1, problem[1])
|
||||
p = ', '.join(problem[2])
|
||||
it.setText(2, p)
|
||||
tl.addChild(it)
|
||||
plaintext.append(','.join([h, problem[0], problem[1], p]))
|
||||
tree.addTopLevelItem(tl)
|
||||
|
||||
t = self.log
|
||||
t.clear()
|
||||
t.setColumnCount(3);
|
||||
t.setHeaderLabels([_('Name'), _('Path from library'), _('Additional Information')])
|
||||
for check in CHECKS:
|
||||
builder(t, checker, check)
|
||||
|
||||
t.setColumnWidth(0, 200)
|
||||
t.setColumnWidth(1, 400)
|
||||
|
||||
self.text_results = '\n'.join(plaintext)
|
||||
|
||||
def copy_to_clipboard(self):
|
||||
QApplication.clipboard().setText(self.text_results)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
app = QApplication([])
|
||||
d = CheckLibraryDialog()
|
||||
d.exec_()
|
||||
@ -10,7 +10,7 @@
|
||||
<x>0</x>
|
||||
<y>0</y>
|
||||
<width>382</width>
|
||||
<height>242</height>
|
||||
<height>265</height>
|
||||
</rect>
|
||||
</property>
|
||||
<property name="windowTitle">
|
||||
|
||||
@ -375,7 +375,7 @@ p, li { white-space: pre-wrap; }
|
||||
<item>
|
||||
<widget class="QLabel" name="label_8">
|
||||
<property name="text">
|
||||
<string>For help with writing advanced news recipes, please visit <a href="http://__appname__.kovidgoyal.net/user_manual/news.html">User Recipes</a></string>
|
||||
<string>For help with writing advanced news recipes, please visit <a href="http://__appname__-ebook.com/user_manual/news.html">User Recipes</a></string>
|
||||
</property>
|
||||
<property name="wordWrap">
|
||||
<bool>true</bool>
|
||||
|
||||
@ -56,6 +56,7 @@ class LocationManager(QObject): # {{{
|
||||
self._mem.append(a)
|
||||
else:
|
||||
ac.setToolTip(tooltip)
|
||||
ac.calibre_name = name
|
||||
|
||||
return ac
|
||||
|
||||
@ -112,7 +113,6 @@ class LocationManager(QObject): # {{{
|
||||
ac.setWhatsThis(t)
|
||||
ac.setStatusTip(t)
|
||||
|
||||
|
||||
@property
|
||||
def has_device(self):
|
||||
return max(self.free) > -1
|
||||
@ -228,6 +228,7 @@ class ToolBar(QToolBar): # {{{
|
||||
self.added_actions = []
|
||||
self.build_bar()
|
||||
self.preferred_width = self.sizeHint().width()
|
||||
self.setAcceptDrops(True)
|
||||
|
||||
def apply_settings(self):
|
||||
sz = gprefs['toolbar_icon_size']
|
||||
@ -317,6 +318,59 @@ class ToolBar(QToolBar): # {{{
|
||||
def database_changed(self, db):
|
||||
pass
|
||||
|
||||
#support drag&drop from/to library from/to reader/card
|
||||
def dragEnterEvent(self, event):
|
||||
md = event.mimeData()
|
||||
if md.hasFormat("application/calibre+from_library") or \
|
||||
md.hasFormat("application/calibre+from_device"):
|
||||
event.setDropAction(Qt.CopyAction)
|
||||
event.accept()
|
||||
else:
|
||||
event.ignore()
|
||||
|
||||
def dragMoveEvent(self, event):
|
||||
allowed = False
|
||||
md = event.mimeData()
|
||||
#Drop is only allowed in the location manager widget's different from the selected one
|
||||
for ac in self.location_manager.available_actions:
|
||||
w = self.widgetForAction(ac)
|
||||
if w is not None:
|
||||
if ( md.hasFormat("application/calibre+from_library") or \
|
||||
md.hasFormat("application/calibre+from_device") ) and \
|
||||
w.geometry().contains(event.pos()) and \
|
||||
isinstance(w, QToolButton) and not w.isChecked():
|
||||
allowed = True
|
||||
break
|
||||
if allowed:
|
||||
event.acceptProposedAction()
|
||||
else:
|
||||
event.ignore()
|
||||
|
||||
def dropEvent(self, event):
|
||||
data = event.mimeData()
|
||||
|
||||
mime = 'application/calibre+from_library'
|
||||
if data.hasFormat(mime):
|
||||
ids = list(map(int, str(data.data(mime)).split()))
|
||||
tgt = None
|
||||
for ac in self.location_manager.available_actions:
|
||||
w = self.widgetForAction(ac)
|
||||
if w is not None and w.geometry().contains(event.pos()):
|
||||
tgt = ac.calibre_name
|
||||
if tgt is not None:
|
||||
if tgt == 'main':
|
||||
tgt = None
|
||||
self.gui.sync_to_device(tgt, False, send_ids=ids)
|
||||
event.accept()
|
||||
|
||||
mime = 'application/calibre+from_device'
|
||||
if data.hasFormat(mime):
|
||||
paths = [unicode(u.toLocalFile()) for u in data.urls()]
|
||||
if paths:
|
||||
self.gui.iactions['Add Books'].add_books_from_device(
|
||||
self.gui.current_view(), paths=paths)
|
||||
event.accept()
|
||||
|
||||
# }}}
|
||||
|
||||
class MainWindowMixin(object): # {{{
|
||||
|
||||
@ -153,7 +153,7 @@ class TextDelegate(QStyledItemDelegate): # {{{
|
||||
complete_items = [i[1] for i in self.auto_complete_function()]
|
||||
completer = QCompleter(complete_items, self)
|
||||
completer.setCaseSensitivity(Qt.CaseInsensitive)
|
||||
completer.setCompletionMode(QCompleter.InlineCompletion)
|
||||
completer.setCompletionMode(QCompleter.PopupCompletion)
|
||||
editor.setCompleter(completer)
|
||||
return editor
|
||||
#}}}
|
||||
|
||||
@ -361,13 +361,14 @@ class BooksModel(QAbstractTableModel): # {{{
|
||||
self.cover_cache.set_cache(ids)
|
||||
|
||||
def current_changed(self, current, previous, emit_signal=True):
|
||||
idx = current.row()
|
||||
self.set_cache(idx)
|
||||
data = self.get_book_display_info(idx)
|
||||
if emit_signal:
|
||||
self.new_bookdisplay_data.emit(data)
|
||||
else:
|
||||
return data
|
||||
if current.isValid():
|
||||
idx = current.row()
|
||||
self.set_cache(idx)
|
||||
data = self.get_book_display_info(idx)
|
||||
if emit_signal:
|
||||
self.new_bookdisplay_data.emit(data)
|
||||
else:
|
||||
return data
|
||||
|
||||
def get_book_info(self, index):
|
||||
if isinstance(index, int):
|
||||
@ -1081,12 +1082,11 @@ class DeviceBooksModel(BooksModel): # {{{
|
||||
self.db = db
|
||||
self.map = list(range(0, len(db)))
|
||||
|
||||
def current_changed(self, current, previous):
|
||||
data = {}
|
||||
item = self.db[self.map[current.row()]]
|
||||
def cover(self, row):
|
||||
item = self.db[self.map[row]]
|
||||
cdata = item.thumbnail
|
||||
img = QImage()
|
||||
if cdata is not None:
|
||||
img = QImage()
|
||||
if hasattr(cdata, 'image_path'):
|
||||
img.load(cdata.image_path)
|
||||
elif cdata:
|
||||
@ -1094,9 +1094,16 @@ class DeviceBooksModel(BooksModel): # {{{
|
||||
img.loadFromData(cdata[-1])
|
||||
else:
|
||||
img.loadFromData(cdata)
|
||||
if img.isNull():
|
||||
img = self.default_image
|
||||
data['cover'] = img
|
||||
if img.isNull():
|
||||
img = self.default_image
|
||||
return img
|
||||
|
||||
def current_changed(self, current, previous):
|
||||
data = {}
|
||||
item = self.db[self.map[current.row()]]
|
||||
cover = self.cover(current.row())
|
||||
if cover is not self.default_image:
|
||||
data['cover'] = cover
|
||||
type = _('Unknown')
|
||||
ext = os.path.splitext(item.path)[1]
|
||||
if ext:
|
||||
|
||||
@ -9,7 +9,8 @@ import os
|
||||
from functools import partial
|
||||
|
||||
from PyQt4.Qt import QTableView, Qt, QAbstractItemView, QMenu, pyqtSignal, \
|
||||
QModelIndex, QIcon, QItemSelection
|
||||
QModelIndex, QIcon, QItemSelection, QMimeData, QDrag, QApplication, \
|
||||
QPoint, QPixmap, QUrl, QImage, QPainter, QColor, QRect
|
||||
|
||||
from calibre.gui2.library.delegates import RatingDelegate, PubDateDelegate, \
|
||||
TextDelegate, DateDelegate, TagsDelegate, CcTextDelegate, \
|
||||
@ -18,7 +19,8 @@ from calibre.gui2.library.models import BooksModel, DeviceBooksModel
|
||||
from calibre.utils.config import tweaks, prefs
|
||||
from calibre.gui2 import error_dialog, gprefs
|
||||
from calibre.gui2.library import DEFAULT_SORT
|
||||
|
||||
from calibre.constants import filesystem_encoding
|
||||
from calibre import force_unicode
|
||||
|
||||
class BooksView(QTableView): # {{{
|
||||
|
||||
@ -31,6 +33,7 @@ class BooksView(QTableView): # {{{
|
||||
self.setDragEnabled(True)
|
||||
self.setDragDropOverwriteMode(False)
|
||||
self.setDragDropMode(self.DragDrop)
|
||||
self.drag_start_pos = None
|
||||
self.setAlternatingRowColors(True)
|
||||
self.setSelectionBehavior(self.SelectRows)
|
||||
self.setShowGrid(False)
|
||||
@ -422,10 +425,92 @@ class BooksView(QTableView): # {{{
|
||||
Accept a drop event and return a list of paths that can be read from
|
||||
and represent files with extensions.
|
||||
'''
|
||||
if event.mimeData().hasFormat('text/uri-list'):
|
||||
urls = [unicode(u.toLocalFile()) for u in event.mimeData().urls()]
|
||||
md = event.mimeData()
|
||||
if md.hasFormat('text/uri-list') and not \
|
||||
md.hasFormat('application/calibre+from_library'):
|
||||
urls = [unicode(u.toLocalFile()) for u in md.urls()]
|
||||
return [u for u in urls if os.path.splitext(u)[1] and os.access(u, os.R_OK)]
|
||||
|
||||
def drag_icon(self, cover, multiple):
|
||||
cover = cover.scaledToHeight(120, Qt.SmoothTransformation)
|
||||
if multiple:
|
||||
base_width = cover.width()
|
||||
base_height = cover.height()
|
||||
base = QImage(base_width+21, base_height+21,
|
||||
QImage.Format_ARGB32_Premultiplied)
|
||||
base.fill(QColor(255, 255, 255, 0).rgba())
|
||||
p = QPainter(base)
|
||||
rect = QRect(20, 0, base_width, base_height)
|
||||
p.fillRect(rect, QColor('white'))
|
||||
p.drawRect(rect)
|
||||
rect.moveLeft(10)
|
||||
rect.moveTop(10)
|
||||
p.fillRect(rect, QColor('white'))
|
||||
p.drawRect(rect)
|
||||
rect.moveLeft(0)
|
||||
rect.moveTop(20)
|
||||
p.fillRect(rect, QColor('white'))
|
||||
p.save()
|
||||
p.setCompositionMode(p.CompositionMode_SourceAtop)
|
||||
p.drawImage(rect.topLeft(), cover)
|
||||
p.restore()
|
||||
p.drawRect(rect)
|
||||
p.end()
|
||||
cover = base
|
||||
return QPixmap.fromImage(cover)
|
||||
|
||||
def drag_data(self):
|
||||
m = self.model()
|
||||
db = m.db
|
||||
rows = self.selectionModel().selectedRows()
|
||||
selected = map(m.id, rows)
|
||||
ids = ' '.join(map(str, selected))
|
||||
md = QMimeData()
|
||||
md.setData('application/calibre+from_library', ids)
|
||||
fmt = prefs['output_format']
|
||||
|
||||
def url_for_id(i):
|
||||
ans = db.format_abspath(i, fmt, index_is_id=True)
|
||||
if ans is None:
|
||||
fmts = db.formats(i, index_is_id=True)
|
||||
if fmts:
|
||||
fmts = fmts.split(',')
|
||||
else:
|
||||
fmts = []
|
||||
for f in fmts:
|
||||
ans = db.format_abspath(i, f, index_is_id=True)
|
||||
if ans is not None:
|
||||
break
|
||||
if ans is None:
|
||||
ans = db.abspath(i, index_is_id=True)
|
||||
return QUrl.fromLocalFile(ans)
|
||||
|
||||
md.setUrls([url_for_id(i) for i in selected])
|
||||
drag = QDrag(self)
|
||||
drag.setMimeData(md)
|
||||
cover = self.drag_icon(m.cover(self.currentIndex().row()),
|
||||
len(selected) > 1)
|
||||
drag.setHotSpot(QPoint(cover.width()//3, cover.height()//3))
|
||||
drag.setPixmap(cover)
|
||||
return drag
|
||||
|
||||
def mousePressEvent(self, event):
|
||||
if event.button() == Qt.LeftButton:
|
||||
self.drag_start_pos = event.pos()
|
||||
return QTableView.mousePressEvent(self, event)
|
||||
|
||||
def mouseMoveEvent(self, event):
|
||||
if not (event.buttons() & Qt.LeftButton) or self.drag_start_pos is None:
|
||||
return
|
||||
if (event.pos() - self.drag_start_pos).manhattanLength() \
|
||||
< QApplication.startDragDistance():
|
||||
return
|
||||
index = self.indexAt(event.pos())
|
||||
if not index.isValid():
|
||||
return
|
||||
drag = self.drag_data()
|
||||
drag.exec_(Qt.CopyAction)
|
||||
|
||||
def dragEnterEvent(self, event):
|
||||
if int(event.possibleActions() & Qt.CopyAction) + \
|
||||
int(event.possibleActions() & Qt.MoveAction) == 0:
|
||||
@ -547,6 +632,21 @@ class DeviceBooksView(BooksView): # {{{
|
||||
self.setDragDropMode(self.NoDragDrop)
|
||||
self.setAcceptDrops(False)
|
||||
|
||||
def drag_data(self):
|
||||
m = self.model()
|
||||
rows = self.selectionModel().selectedRows()
|
||||
paths = [force_unicode(p, enc=filesystem_encoding) for p in m.paths(rows) if p]
|
||||
md = QMimeData()
|
||||
md.setData('application/calibre+from_device', 'dummy')
|
||||
md.setUrls([QUrl.fromLocalFile(p) for p in paths])
|
||||
drag = QDrag(self)
|
||||
drag.setMimeData(md)
|
||||
cover = self.drag_icon(m.cover(self.currentIndex().row()), len(paths) >
|
||||
1)
|
||||
drag.setHotSpot(QPoint(cover.width()//3, cover.height()//3))
|
||||
drag.setPixmap(cover)
|
||||
return drag
|
||||
|
||||
def contextMenuEvent(self, event):
|
||||
edit_collections = callable(getattr(self._model.db, 'supports_collections', None)) and \
|
||||
self._model.db.supports_collections() and \
|
||||
|
||||
@ -7,6 +7,7 @@ __docformat__ = 'restructuredtext en'
|
||||
|
||||
from PyQt4.Qt import QProgressDialog, QThread, Qt, pyqtSignal
|
||||
|
||||
from calibre.gui2.dialogs.check_library import CheckLibraryDialog
|
||||
from calibre.gui2.preferences import ConfigWidgetBase, test_widget
|
||||
from calibre.gui2.preferences.misc_ui import Ui_Form
|
||||
from calibre.gui2 import error_dialog, config, warning_dialog, \
|
||||
@ -89,6 +90,7 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
|
||||
self.device_detection_button.clicked.connect(self.debug_device_detection)
|
||||
self.compact_button.clicked.connect(self.compact)
|
||||
self.button_all_books_dirty.clicked.connect(self.mark_dirty)
|
||||
self.button_check_library.clicked.connect(self.check_library)
|
||||
self.button_open_config_dir.clicked.connect(self.open_config_dir)
|
||||
self.button_osx_symlinks.clicked.connect(self.create_symlinks)
|
||||
self.button_osx_symlinks.setVisible(isosx)
|
||||
@ -100,6 +102,11 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
|
||||
_('Metadata will be backed up while calibre is running, at the '
|
||||
'rate of 30 books per minute.'), show=True)
|
||||
|
||||
def check_library(self):
|
||||
db = self.gui.library_view.model().db
|
||||
d = CheckLibraryDialog(self.gui.parent(), db)
|
||||
d.exec_()
|
||||
|
||||
def debug_device_detection(self, *args):
|
||||
from calibre.gui2.preferences.device_debug import DebugDevice
|
||||
d = DebugDevice(self)
|
||||
|
||||
@ -131,6 +131,13 @@
|
||||
</property>
|
||||
</widget>
|
||||
</item>
|
||||
<item row="11" column="0" colspan="2">
|
||||
<widget class="QPushButton" name="button_check_library">
|
||||
<property name="text">
|
||||
<string>Check the library folders for potential problems</string>
|
||||
</property>
|
||||
</widget>
|
||||
</item>
|
||||
<item row="20" column="0">
|
||||
<spacer name="verticalSpacer_9">
|
||||
<property name="orientation">
|
||||
|
||||
@ -39,6 +39,12 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
|
||||
|
||||
ConfigWidgetBase.initialize(self)
|
||||
|
||||
if self.gui.device_manager.connected_device is not None:
|
||||
self.device_label.setText(_('Device currently connected: ') +
|
||||
self.gui.device_manager.connected_device.__class__.__name__)
|
||||
else:
|
||||
self.device_label.setText(_('Device currently connected: None'))
|
||||
|
||||
self.devices = ['']
|
||||
for device in device_plugins():
|
||||
n = device.__class__.__name__
|
||||
@ -54,6 +60,7 @@ class ConfigWidget(ConfigWidgetBase, Ui_Form):
|
||||
for w in metadata_writers():
|
||||
for f in w.file_types:
|
||||
self.formats.append(f)
|
||||
self.formats.append('device_db')
|
||||
self.formats.sort()
|
||||
self.formats.insert(1, plugboard_any_format_value)
|
||||
self.new_format.addItems(self.formats)
|
||||
|
||||
@ -19,7 +19,7 @@
|
||||
<property name="text">
|
||||
<string>Here you can change the metadata calibre uses to update a book when saving to disk or sending to device.
|
||||
|
||||
Use this dialog to define a 'plugboard' for a format (or all formats) and a device (or all devices). The plugboard spefies what template is connected to what field. The template is used to compute a value, and that value is assigned to the connected field.
|
||||
Use this dialog to define a 'plugboard' for a format (or all formats) and a device (or all devices). The plugboard specifies what template is connected to what field. The template is used to compute a value, and that value is assigned to the connected field.
|
||||
|
||||
Often templates will contain simple references to composite columns, but this is not necessary. You can use any template in a source box that you can use elsewhere in calibre.
|
||||
|
||||
@ -40,7 +40,18 @@ One possible use for a plugboard is to alter the title to contain series informa
|
||||
</property>
|
||||
</widget>
|
||||
</item>
|
||||
<item row="2" column="0">
|
||||
<item row="2" column="0" colspan="2">
|
||||
<widget class="QLabel" name="device_label">
|
||||
</widget>
|
||||
</item>
|
||||
<item row="3" column="0" colspan="2">
|
||||
<widget class="Line" name="line">
|
||||
<property name="orientation">
|
||||
<enum>Qt::Horizontal</enum>
|
||||
</property>
|
||||
</widget>
|
||||
</item>
|
||||
<item row="4" column="0">
|
||||
<layout class="QGridLayout" name="gridLayout_2">
|
||||
<item row="0" column="1">
|
||||
<widget class="QLabel" name="label_6">
|
||||
@ -123,7 +134,7 @@ One possible use for a plugboard is to alter the title to contain series informa
|
||||
</item>
|
||||
</layout>
|
||||
</item>
|
||||
<item row="2" column="1">
|
||||
<item row="4" column="1">
|
||||
<layout class="QGridLayout" name="fields_layout">
|
||||
<item row="0" column="0">
|
||||
<widget class="QLabel" name="label_2">
|
||||
|
||||
@ -79,6 +79,8 @@ class TagsView(QTreeView): # {{{
|
||||
self.setHeaderHidden(True)
|
||||
self.setItemDelegate(TagDelegate(self))
|
||||
self.made_connections = False
|
||||
self.setAcceptDrops(True)
|
||||
self.setDropIndicatorShown(True)
|
||||
|
||||
def set_database(self, db, tag_match, sort_by):
|
||||
self.hidden_categories = config['tag_browser_hidden_categories']
|
||||
@ -104,6 +106,49 @@ class TagsView(QTreeView): # {{{
|
||||
def database_changed(self, event, ids):
|
||||
self.refresh_required.emit()
|
||||
|
||||
def dragEnterEvent(self, event):
|
||||
md = event.mimeData()
|
||||
if md.hasFormat("application/calibre+from_library"):
|
||||
event.setDropAction(Qt.CopyAction)
|
||||
event.accept()
|
||||
else:
|
||||
event.ignore()
|
||||
|
||||
def dragMoveEvent(self, event):
|
||||
allowed = False
|
||||
idx = self.indexAt(event.pos())
|
||||
m = self.model()
|
||||
p = m.parent(idx)
|
||||
if idx.isValid() and p.isValid():
|
||||
item = m.data(p, Qt.UserRole)
|
||||
if item.type == TagTreeItem.CATEGORY and \
|
||||
item.category_key in \
|
||||
('tags', 'series', 'authors', 'rating', 'publisher'):
|
||||
allowed = True
|
||||
if allowed:
|
||||
event.acceptProposedAction()
|
||||
else:
|
||||
event.ignore()
|
||||
|
||||
def dropEvent(self, event):
|
||||
idx = self.indexAt(event.pos())
|
||||
m = self.model()
|
||||
p = m.parent(idx)
|
||||
if idx.isValid() and p.isValid():
|
||||
item = m.data(p, Qt.UserRole)
|
||||
if item.type == TagTreeItem.CATEGORY and \
|
||||
item.category_key in \
|
||||
('tags', 'series', 'authors', 'rating', 'publisher'):
|
||||
child = m.data(idx, Qt.UserRole)
|
||||
md = event.mimeData()
|
||||
mime = 'application/calibre+from_library'
|
||||
ids = list(map(int, str(md.data(mime)).split()))
|
||||
self.handle_drop(item, child, ids)
|
||||
event.accept()
|
||||
|
||||
def handle_drop(self, parent, child, ids):
|
||||
print 'Dropped ids:', ids
|
||||
|
||||
@property
|
||||
def match_all(self):
|
||||
return self.tag_match and self.tag_match.currentIndex() > 0
|
||||
@ -326,6 +371,8 @@ class TagTreeItem(object): # {{{
|
||||
self.children.append(child)
|
||||
|
||||
def data(self, role):
|
||||
if role == Qt.UserRole:
|
||||
return self
|
||||
if self.type == self.TAG:
|
||||
return self.tag_data(role)
|
||||
if self.type == self.CATEGORY:
|
||||
@ -544,8 +591,14 @@ class TagsModel(QAbstractItemModel): # {{{
|
||||
def headerData(self, *args):
|
||||
return NONE
|
||||
|
||||
def flags(self, *args):
|
||||
return Qt.ItemIsEnabled|Qt.ItemIsSelectable|Qt.ItemIsEditable
|
||||
def flags(self, index, *args):
|
||||
ans = Qt.ItemIsEnabled|Qt.ItemIsSelectable|Qt.ItemIsEditable
|
||||
if index.isValid() and self.parent(index).isValid():
|
||||
ans |= Qt.ItemIsDropEnabled
|
||||
return ans
|
||||
|
||||
def supportedDropActions(self):
|
||||
return Qt.CopyAction|Qt.MoveAction
|
||||
|
||||
def path_for_index(self, index):
|
||||
ans = []
|
||||
|
||||
189
src/calibre/library/check_library.py
Normal file
189
src/calibre/library/check_library.py
Normal file
@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
|
||||
__docformat__ = 'restructuredtext en'
|
||||
|
||||
import re, os, traceback
|
||||
|
||||
from calibre import isbytestring
|
||||
from calibre.constants import filesystem_encoding
|
||||
from calibre.ebooks import BOOK_EXTENSIONS
|
||||
|
||||
EBOOK_EXTENSIONS = frozenset(BOOK_EXTENSIONS)
|
||||
NORMALS = frozenset(['metadata.opf', 'cover.jpg'])
|
||||
|
||||
CHECKS = [('invalid_titles', _('Invalid titles')),
|
||||
('extra_titles', _('Extra titles')),
|
||||
('invalid_authors', _('Invalid authors')),
|
||||
('extra_authors', _('Extra authors')),
|
||||
('missing_formats', _('Missing book formats')),
|
||||
('extra_formats', _('Extra book formats')),
|
||||
('extra_files', _('Unknown files in books')),
|
||||
('failed_folders', _('Folders raising exception'))
|
||||
]
|
||||
|
||||
|
||||
class CheckLibrary(object):
|
||||
|
||||
def __init__(self, library_path, db):
|
||||
if isbytestring(library_path):
|
||||
library_path = library_path.decode(filesystem_encoding)
|
||||
self.src_library_path = os.path.abspath(library_path)
|
||||
self.db = db
|
||||
|
||||
self.is_case_sensitive = db.is_case_sensitive
|
||||
|
||||
self.all_authors = frozenset([x[1] for x in db.all_authors()])
|
||||
self.all_ids = frozenset([id for id in db.all_ids()])
|
||||
self.all_dbpaths = frozenset(self.dbpath(id) for id in self.all_ids)
|
||||
self.all_lc_dbpaths = frozenset([f.lower() for f in self.all_dbpaths])
|
||||
|
||||
self.db_id_regexp = re.compile(r'^.* \((\d+)\)$')
|
||||
self.bad_ext_pat = re.compile(r'[^a-z0-9]+')
|
||||
|
||||
self.dirs = []
|
||||
self.book_dirs = []
|
||||
|
||||
self.potential_authors = {}
|
||||
self.invalid_authors = []
|
||||
self.extra_authors = []
|
||||
|
||||
self.invalid_titles = []
|
||||
self.extra_titles = []
|
||||
|
||||
self.unknown_book_files = []
|
||||
self.missing_formats = []
|
||||
self.extra_formats = []
|
||||
self.extra_files = []
|
||||
|
||||
|
||||
def dbpath(self, id):
|
||||
return self.db.path(id, index_is_id=True)
|
||||
|
||||
@property
|
||||
def errors_occurred(self):
|
||||
return self.failed_folders or self.mismatched_dirs or \
|
||||
self.conflicting_custom_cols or self.failed_restores
|
||||
|
||||
def scan_library(self, name_ignores, extension_ignores):
|
||||
self.ignore_names = frozenset(name_ignores)
|
||||
self.ignore_ext = frozenset(['.'+ e for e in extension_ignores])
|
||||
|
||||
lib = self.src_library_path
|
||||
for auth_dir in os.listdir(lib):
|
||||
if auth_dir in self.ignore_names or auth_dir == 'metadata.db':
|
||||
continue
|
||||
auth_path = os.path.join(lib, auth_dir)
|
||||
# First check: author must be a directory
|
||||
if not os.path.isdir(auth_path):
|
||||
self.invalid_authors.append((auth_dir, auth_dir, []))
|
||||
continue
|
||||
|
||||
self.potential_authors[auth_dir] = {}
|
||||
|
||||
# Look for titles in the author directories
|
||||
found_titles = False
|
||||
for title_dir in os.listdir(auth_path):
|
||||
if title_dir in self.ignore_names:
|
||||
continue
|
||||
title_path = os.path.join(auth_path, title_dir)
|
||||
db_path = os.path.join(auth_dir, title_dir)
|
||||
m = self.db_id_regexp.search(title_dir)
|
||||
# Second check: title must have an ID and must be a directory
|
||||
if m is None or not os.path.isdir(title_path):
|
||||
self.invalid_titles.append((auth_dir, db_path, [title_dir]))
|
||||
continue
|
||||
|
||||
id = m.group(1)
|
||||
# Third check: the id must be in the DB and the paths must match
|
||||
if self.is_case_sensitive:
|
||||
if int(id) not in self.all_ids or \
|
||||
db_path not in self.all_dbpaths:
|
||||
self.extra_titles.append((title_dir, db_path, []))
|
||||
continue
|
||||
else:
|
||||
if int(id) not in self.all_ids or \
|
||||
db_path.lower() not in self.all_lc_dbpaths:
|
||||
self.extra_titles.append((title_dir, db_path, []))
|
||||
continue
|
||||
|
||||
# Record the book to check its formats
|
||||
self.book_dirs.append((db_path, title_dir, id))
|
||||
found_titles = True
|
||||
|
||||
# Fourth check: author directories that contain no titles
|
||||
if not found_titles:
|
||||
self.extra_authors.append((auth_dir, auth_dir, []))
|
||||
|
||||
for x in self.book_dirs:
|
||||
try:
|
||||
self.process_book(lib, x)
|
||||
except:
|
||||
traceback.print_exc()
|
||||
# Sort-of check: exception processing directory
|
||||
self.failed_folders.append((title_path, traceback.format_exc(), []))
|
||||
|
||||
def is_ebook_file(self, filename):
|
||||
ext = os.path.splitext(filename)[1]
|
||||
if not ext:
|
||||
return False
|
||||
ext = ext[1:].lower()
|
||||
if ext in EBOOK_EXTENSIONS:
|
||||
return True
|
||||
if self.bad_ext_pat.search(ext) is not None:
|
||||
return False
|
||||
return True
|
||||
|
||||
def process_book(self, lib, book_info):
|
||||
(db_path, title_dir, book_id) = book_info
|
||||
filenames = frozenset([f for f in os.listdir(os.path.join(lib, db_path))
|
||||
if os.path.splitext(f)[1] not in self.ignore_ext])
|
||||
book_id = int(book_id)
|
||||
formats = frozenset(filter(self.is_ebook_file, filenames))
|
||||
book_formats = frozenset([x[0]+'.'+x[1].lower() for x in
|
||||
self.db.format_files(book_id, index_is_id=True)])
|
||||
|
||||
if self.is_case_sensitive:
|
||||
unknowns = frozenset(filenames-formats-NORMALS)
|
||||
# Check: any books that aren't formats or normally there?
|
||||
if unknowns:
|
||||
self.extra_files.append((title_dir, db_path, unknowns))
|
||||
|
||||
# Check: any book formats that should be there?
|
||||
missing = book_formats - formats
|
||||
if missing:
|
||||
self.missing_formats.append((title_dir, db_path, missing))
|
||||
|
||||
# Check: any book formats that shouldn't be there?
|
||||
extra = formats - book_formats - NORMALS
|
||||
if extra:
|
||||
self.extra_formats.append((title_dir, db_path, extra))
|
||||
else:
|
||||
def lc_map(fnames, fset):
|
||||
m = {}
|
||||
for f in fnames:
|
||||
m[f.lower()] = f
|
||||
return [m[f] for f in fset]
|
||||
|
||||
filenames_lc = frozenset([f.lower() for f in filenames])
|
||||
formats_lc = frozenset([f.lower() for f in formats])
|
||||
unknowns = frozenset(filenames_lc-formats_lc-NORMALS)
|
||||
# Check: any books that aren't formats or normally there?
|
||||
if unknowns:
|
||||
self.extra_files.append((title_dir, db_path,
|
||||
lc_map(filenames, unknowns)))
|
||||
|
||||
book_formats_lc = frozenset([f.lower() for f in book_formats])
|
||||
# Check: any book formats that should be there?
|
||||
missing = book_formats_lc - formats_lc
|
||||
if missing:
|
||||
self.missing_formats.append((title_dir, db_path,
|
||||
lc_map(book_formats, missing)))
|
||||
|
||||
# Check: any book formats that shouldn't be there?
|
||||
extra = formats_lc - book_formats_lc - NORMALS
|
||||
if extra:
|
||||
self.extra_formats.append((title_dir, db_path,
|
||||
lc_map(formats, extra)))
|
||||
@ -302,7 +302,7 @@ def do_add_empty(db, title, authors, isbn):
|
||||
if isbn:
|
||||
mi.isbn = isbn
|
||||
db.import_book(mi, [])
|
||||
write_dirtied()
|
||||
write_dirtied(db)
|
||||
send_message()
|
||||
|
||||
def command_add(args, dbpath):
|
||||
@ -456,7 +456,7 @@ def do_set_metadata(db, id, stream):
|
||||
db.set_metadata(id, mi)
|
||||
db.clean()
|
||||
do_show_metadata(db, id, False)
|
||||
write_dirtied()
|
||||
write_dirtied(db)
|
||||
send_message()
|
||||
|
||||
def set_metadata_option_parser():
|
||||
@ -874,11 +874,93 @@ def command_saved_searches(args, dbpath):
|
||||
|
||||
return 0
|
||||
|
||||
def check_library_option_parser():
|
||||
from calibre.library.check_library import CHECKS
|
||||
parser = get_parser(_('''\
|
||||
%prog check_library [options]
|
||||
|
||||
Perform some checks on the filesystem representing a library. Reports are {0}
|
||||
''').format(', '.join([c[0] for c in CHECKS])))
|
||||
|
||||
parser.add_option('-c', '--csv', default=False, action='store_true',
|
||||
help=_('Output in CSV'))
|
||||
|
||||
parser.add_option('-r', '--report', default=None, dest='report',
|
||||
help=_("Comma-separated list of reports.\n"
|
||||
"Default: all"))
|
||||
|
||||
parser.add_option('-e', '--ignore_extensions', default=None, dest='exts',
|
||||
help=_("Comma-separated list of extensions to ignore.\n"
|
||||
"Default: all"))
|
||||
|
||||
parser.add_option('-n', '--ignore_names', default=None, dest='names',
|
||||
help=_("Comma-separated list of names to ignore.\n"
|
||||
"Default: all"))
|
||||
return parser
|
||||
|
||||
def command_check_library(args, dbpath):
|
||||
from calibre.library.check_library import CheckLibrary, CHECKS
|
||||
parser = check_library_option_parser()
|
||||
opts, args = parser.parse_args(args)
|
||||
if len(args) != 0:
|
||||
parser.print_help()
|
||||
return 1
|
||||
|
||||
if opts.library_path is not None:
|
||||
dbpath = opts.library_path
|
||||
|
||||
if isbytestring(dbpath):
|
||||
dbpath = dbpath.decode(preferred_encoding)
|
||||
|
||||
if opts.report is None:
|
||||
checks = CHECKS
|
||||
else:
|
||||
checks = []
|
||||
for r in opts.report.split(','):
|
||||
found = False
|
||||
for c in CHECKS:
|
||||
if c[0] == r:
|
||||
checks.append(c)
|
||||
found = True
|
||||
break
|
||||
if not found:
|
||||
print _('Unknown report check'), r
|
||||
return 1
|
||||
|
||||
if opts.names is None:
|
||||
names = []
|
||||
else:
|
||||
names = [f.strip() for f in opts.names.split(',') if f.strip()]
|
||||
if opts.exts is None:
|
||||
exts = []
|
||||
else:
|
||||
exts = [f.strip() for f in opts.exts.split(',') if f.strip()]
|
||||
|
||||
def print_one(checker, check):
|
||||
attr = check[0]
|
||||
list = getattr(checker, attr, None)
|
||||
if list is None:
|
||||
return
|
||||
if opts.csv:
|
||||
for i in list:
|
||||
print check[1] + ',' + i[0] + ',' + i[1] + ',' + '|'.join(i[2])
|
||||
else:
|
||||
print check[1]
|
||||
for i in list:
|
||||
print ' %-30.30s - %-30.30s - %s'%(i[0], i[1], ', '.join(i[2]))
|
||||
|
||||
db = LibraryDatabase2(dbpath)
|
||||
checker = CheckLibrary(dbpath, db)
|
||||
checker.scan_library(names, exts)
|
||||
for check in checks:
|
||||
print_one(checker, check)
|
||||
|
||||
|
||||
COMMANDS = ('list', 'add', 'remove', 'add_format', 'remove_format',
|
||||
'show_metadata', 'set_metadata', 'export', 'catalog',
|
||||
'saved_searches', 'add_custom_column', 'custom_columns',
|
||||
'remove_custom_column', 'set_custom', 'restore_database')
|
||||
'remove_custom_column', 'set_custom', 'restore_database',
|
||||
'check_library')
|
||||
|
||||
def restore_database_option_parser():
|
||||
parser = get_parser(_(
|
||||
@ -889,7 +971,7 @@ def restore_database_option_parser():
|
||||
files in each directory of the calibre library. This is
|
||||
useful if your metadata.db file has been corrupted.
|
||||
|
||||
WARNING: This completely regenrates your datbase. You will
|
||||
WARNING: This completely regenerates your datbase. You will
|
||||
lose stored per-book conversion settings and custom recipes.
|
||||
'''))
|
||||
return parser
|
||||
|
||||
@ -850,6 +850,15 @@ class LibraryDatabase2(LibraryDatabase, SchemaUpgrade, CustomColumns):
|
||||
return set([])
|
||||
return set([f[0] for f in formats])
|
||||
|
||||
def format_files(self, index, index_is_id=False):
|
||||
id = index if index_is_id else self.id(index)
|
||||
try:
|
||||
formats = self.conn.get('SELECT name,format FROM data WHERE book=?', (id,))
|
||||
formats = map(lambda x:(x[0], x[1]), formats)
|
||||
return formats
|
||||
except:
|
||||
return []
|
||||
|
||||
def formats(self, index, index_is_id=False, verify_formats=True):
|
||||
''' Return available formats as a comma separated list or None if there are no available formats '''
|
||||
id = index if index_is_id else self.id(index)
|
||||
@ -1584,7 +1593,7 @@ class LibraryDatabase2(LibraryDatabase, SchemaUpgrade, CustomColumns):
|
||||
self.conn.commit()
|
||||
|
||||
def delete_publisher_using_id(self, old_id):
|
||||
self.dirty_books_referencing('publisher', id, commit=False)
|
||||
self.dirty_books_referencing('publisher', old_id, commit=False)
|
||||
self.conn.execute('''DELETE FROM books_publishers_link
|
||||
WHERE publisher=?''', (old_id,))
|
||||
self.conn.execute('DELETE FROM publishers WHERE id=?', (old_id,))
|
||||
|
||||
@ -39,7 +39,7 @@ class Restore(Thread):
|
||||
self.src_library_path = os.path.abspath(library_path)
|
||||
self.progress_callback = progress_callback
|
||||
self.db_id_regexp = re.compile(r'^.* \((\d+)\)$')
|
||||
self.bad_ext_pat = re.compile(r'[^a-z]+')
|
||||
self.bad_ext_pat = re.compile(r'[^a-z0-9]+')
|
||||
if not callable(self.progress_callback):
|
||||
self.progress_callback = lambda x, y: x
|
||||
self.dirs = []
|
||||
|
||||
@ -8,6 +8,7 @@ __docformat__ = 'restructuredtext en'
|
||||
|
||||
import os, traceback, cStringIO, re
|
||||
|
||||
from calibre.constants import DEBUG
|
||||
from calibre.utils.config import Config, StringConfig, tweaks
|
||||
from calibre.utils.formatter import TemplateFormatter
|
||||
from calibre.utils.filenames import shorten_components_to, supports_long_names, \
|
||||
@ -17,7 +18,7 @@ from calibre.ebooks.metadata.meta import set_metadata
|
||||
from calibre.constants import preferred_encoding, filesystem_encoding
|
||||
from calibre.ebooks.metadata import fmt_sidx
|
||||
from calibre.ebooks.metadata import title_sort
|
||||
from calibre import strftime
|
||||
from calibre import strftime, prints
|
||||
|
||||
plugboard_any_device_value = 'any device'
|
||||
plugboard_any_format_value = 'any format'
|
||||
@ -114,8 +115,14 @@ class SafeFormat(TemplateFormatter):
|
||||
|
||||
def get_value(self, key, args, kwargs):
|
||||
try:
|
||||
b = self.book.get_user_metadata(key, False)
|
||||
key = key.lower()
|
||||
try:
|
||||
b = self.book.get_user_metadata(key, False)
|
||||
except:
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
b = None
|
||||
|
||||
if b is not None and b['datatype'] == 'composite':
|
||||
if key in self.composite_values:
|
||||
return self.composite_values[key]
|
||||
@ -123,11 +130,13 @@ class SafeFormat(TemplateFormatter):
|
||||
self.composite_values[key] = \
|
||||
self.vformat(b['display']['composite_template'], [], kwargs)
|
||||
return self.composite_values[key]
|
||||
if kwargs[key]:
|
||||
return self.sanitize(kwargs[key.lower()])
|
||||
if key in kwargs:
|
||||
return kwargs[key].replace('/', '_').replace('\\', '_')
|
||||
return ''
|
||||
except:
|
||||
return ''
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
return key
|
||||
|
||||
safe_formatter = SafeFormat()
|
||||
|
||||
@ -174,8 +183,8 @@ def get_components(template, mi, id, timefmt='%b %Y', length=250,
|
||||
elif custom_metadata[key]['datatype'] == 'bool':
|
||||
format_args[key] = _('yes') if format_args[key] else _('no')
|
||||
|
||||
components = safe_formatter.safe_format(template, format_args, '', mi,
|
||||
sanitize=sanitize_func)
|
||||
components = safe_formatter.safe_format(template, format_args,
|
||||
'G_C-EXCEPTION!', mi)
|
||||
components = [x.strip() for x in components.split('/') if x.strip()]
|
||||
components = [sanitize_func(x) for x in components if x]
|
||||
if not components:
|
||||
@ -259,7 +268,8 @@ def save_book_to_disk(id, db, root, opts, length):
|
||||
cpb = cpb[dev_name]
|
||||
else:
|
||||
cpb = None
|
||||
#prints('Using plugboard:', fmt, cpb)
|
||||
if DEBUG:
|
||||
prints('Save-to-disk using plugboard:', fmt, cpb)
|
||||
data = db.format(id, fmt, index_is_id=True)
|
||||
if data is None:
|
||||
continue
|
||||
@ -277,7 +287,8 @@ def save_book_to_disk(id, db, root, opts, length):
|
||||
newmi = mi
|
||||
set_metadata(stream, newmi, fmt)
|
||||
except:
|
||||
traceback.print_exc()
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
stream.seek(0)
|
||||
data = stream.read()
|
||||
fmt_path = base_path+'.'+str(fmt)
|
||||
|
||||
@ -4,3 +4,83 @@
|
||||
|
||||
Editing E-book Metadata
|
||||
========================
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
|
||||
E-books come in all shapes and sizes and more often than not, their metadata (things like title/author/series/publisher) is incomplete or incorrect.
|
||||
The simplest way to change metadata in |app| is to simply double click on an entry and type in the correct replacement.
|
||||
For more sophisticated, "power editing" use the edit metadata tools discussed below.
|
||||
|
||||
Editing the metadata of one book at a time
|
||||
-------------------------------------------
|
||||
|
||||
Click the book you want to edit and then click the :guilabel:`Edit metadata` button or press the ``E`` key. A dialog opens that allows you to edit all aspects of the metadata. It has various features to make editing faster and more efficient. A list of the commonly used tips:
|
||||
|
||||
* You can click the button in between title and authors to swap them automatically.
|
||||
* You can click the button next to author sort to automatically to have |app| automatically fill it from the author name.
|
||||
* You can click the button next to tags to use the Tag Editor to manage the tags associated with the book.
|
||||
* The ISBN box will have a red background if you enter an invalid ISBN. It will be green for valid ISBNs
|
||||
* The author sort box will be red if the author sort value differs from what |app| thinks it should be.
|
||||
|
||||
Downloading metadata
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The nicest feature of the edit metadata dialog is its ability to automatically fill in many metadata fields by getting metadata from various websites. Currently, |app| uses isbndb.com, Google Books, Amazon and Library Thing. The metadata download can fill in Title, author, series, tags, rating, description and ISBN for you.
|
||||
|
||||
To use the download, fill in the title and author fields and click the :guilabel:`Fetch metadata` button. |app| will present you with a list of books that most closely match the title and author. If you fill in the ISBN field first, it will be used in preference to the title and author. If no matches are found, try making your search a little less specific by including only some key words in the title and only the author last name.
|
||||
|
||||
Managing book formats
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
In |app|, a single book entry can have many different *formats* associated with it. For example you may have obtained the Complete Works of Shakespeare in EPUB format and later converted it to MOBI to read on your Kindle. |app| automatically manages multiple formats for you. In the :guilabel:`Available formats` section of the Edit metadata dialog, you can manage these formats. You can add a new format, delete an existing format and also ask |app| to set the metadata and cover for the book entry from the metadata in one of the formats.
|
||||
|
||||
All about covers
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You can ask |app| to download book covers for you, provided the book has a known ISBN. Alternatively you can specify a file on your computer to use as the cover. |app| can even generate a default cover with basic metadata on it for you. You can drag and drop images onto the cover to change it and also right click to copy/paste cover images.
|
||||
|
||||
In addition, there is a button to automatically trim borders from the cover, in case your cover image has an ugly border.
|
||||
|
||||
|
||||
Editing the metadata of many books at a time
|
||||
---------------------------------------------
|
||||
|
||||
First select the books you want to edit by holding Ctrl or Shift and clicking on them. If you select more than one book, clicking the :guilabel:`Edit metadata` button will cause a new *Bulk* metadata edit dialog to open. Using this dialog, you can quickly set the author/publisher/rating/tags/series etc of a bunch of books to the same value. This is particularly useful if you have just imported a number of books that have some metadata in common. You can also click the arrow next to the :guilabel:`Edit metadata` button and select :guilabel:`Edit metadata individually` to use the powerful single book edit dialog from above for all the selected books in succession.
|
||||
|
||||
Search and replace
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The Bulk metadata edit dialog allows you to perform arbitrarily powerful search and replace operations on the selected books. By default it uses a simple text search and replace, but it also support *regular expressions*. For more on regular expressions, see :ref:`regexptutorial`.
|
||||
|
||||
As noted above, there are two search and replace modes: character match and regular expression. Character match will look in the `Search field` you choose for the characters you type in the `search for` box and replace those characters with what you type in the `replace with` box. Each occurance of the search characters in the field will be replaced. For example, assume the field being searched contains `a bad cat`. if you search for `a` to be replaced with `HELLO`, then the result will be `HELLO bHELLOd cHELLOt`.
|
||||
|
||||
If the field you are searching on is a `multiple` field like tags, then each tag is treated separately. For example, if your tags contain `Horror, Scary`, the search expression `r,` will not match anything because the expression will first be applied to `Horror` and then to `Scary`.
|
||||
|
||||
If you want the search to ignore upper/lowercase differences, uncheck the `Case sensitive` box.
|
||||
|
||||
You can have |app| change the case of the result (information after the replace has happened) by choosing one of the functions from the `Apply function after replace` box. The operations available are:
|
||||
|
||||
*`Lower case` -- change all the characters in the field to lower case
|
||||
*`Upper case` -- change all the characters in the field to upper case
|
||||
*`Title case` -- capitalize each word in the result.
|
||||
|
||||
The `Your test` box is provided for you to enter text to check that search/replace is doing what you want. In the majority of cases the book test boxes will be sufficient, but it is possible that there is a case you want to check that isn't shown in these boxes. Enter that case into `Your test`.
|
||||
|
||||
Regular expression mode has some differences from character mode, beyond (of course) using regular expressions. The first is that functions are applied to the parts of the string matched by the search string, not the entire field. The second is that functions apply to the replacement string, not to the entire field.
|
||||
|
||||
The third and most important is that the replace string can make reference to parts of the search string by using backreferences. A backreference is ``\\n`` where n is an integer that refers to the n'th parenthesized group in the search expression. For example, given the same example as above, `a bad cat`, a search expression `a (...) (...)`, and a replace expression `a \\2 \\1`, the result will be `a cat bad`. Please see the :ref:`regexptutorial` for more information on backreferences.
|
||||
|
||||
One useful pattern: assume you want to change the case of an entire field. The easiest way to do this is to use character mode, but lets further assume you want to use regular expression mode. The search expression should be `(.*)` the replace expression should be `\1`, and the desired case change function should be selected.
|
||||
|
||||
Finally, in regular expression mode you can copy values from one field to another. Simply make the source and destination field different. The copy can replace the destination field, prepend to the field (add to the front), or append to the field (add at the end). The 'use comma' checkbox tells |app| to (or not to) add a comma between the text and the destination field in prepend and append modes. If the destination is multiple (e.g., tags), then you cannot uncheck this box.
|
||||
|
||||
Search and replace is done after all the other metadata changes in the other tabs are applied. This can lead to some confusion, because the test boxes will show the information before the other changes, but the operation will be applied after the other changes. If you have any doubts about what is going to happen, do not mix search/replace with other changes.
|
||||
|
||||
Bulk downloading of metadata
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If you want to download the metadata for multiple books at once, click the arrow next to the :guilabel:`Edit metadata` button and select :guilabel:`Download metadata and covers`. You can choose to download only metadata, only covers, both or only social metadata (tags/rating/series).
|
||||
|
||||
|
||||
|
||||
@ -108,7 +108,7 @@ Function references appear in the format part, going after the ``:`` and before
|
||||
|
||||
Functions are always applied before format specifications. See further down for an example of using both a format and a function, where this order is demonstrated.
|
||||
|
||||
The syntax for using functions is ``{field:function(arguments)}``, or ``{field:function(arguments)|prefix|suffix}``. Argument values cannot contain a comma, because it is used to separate arguments. The last (or only) argument cannot contain a closing parenthesis ( ')' ). Functions return the value of the field used in the template, suitably modified.
|
||||
The syntax for using functions is ``{field:function(arguments)}``, or ``{field:function(arguments)|prefix|suffix}``. Arguments are separated by commas. Commas inside arguments must be preceeded by a backslash ( '\\' ). The last (or only) argument cannot contain a closing parenthesis ( ')' ). Functions return the value of the field used in the template, suitably modified.
|
||||
|
||||
The functions available are:
|
||||
|
||||
@ -119,6 +119,7 @@ The functions available are:
|
||||
* ``ifempty(text)`` -- if the field is not empty, return the value of the field. Otherwise return `text`.
|
||||
* ``test(text if not empty, text if empty)`` -- return `text if not empty` if the field is not empty, otherwise return `text if empty`.
|
||||
* ``contains(pattern, text if match, text if not match`` -- checks if field contains matches for the regular expression `pattern`. Returns `text if match` if matches are found, otherwise it returns `text if no match`.
|
||||
* ``switch(pattern, value, pattern, value, ..., else_value)`` -- for each ``pattern, value`` pair, checks if the field matches the ``pattern`` and if so, returns that ``value``. If no ``pattern`` matches, then ``else_value`` is returned. You can have as many ``pattern, value`` pairs as you want.
|
||||
* ``shorten(left chars, middle text, right chars)`` -- Return a shortened version of the field, consisting of `left chars` characters from the beginning of the field, followed by `middle text`, followed by `right chars` characters from the end of the string. `Left chars` and `right chars` must be integers. For example, assume the title of the book is `Ancient English Laws in the Times of Ivanhoe`, and you want it to fit in a space of at most 15 characters. If you use ``{title:shorten(9,-,5)}``, the result will be `Ancient E-nhoe`. If the field's length is less than ``left chars`` + ``right chars`` + the length of ``middle text``, then the field will be used intact. For example, the title `The Dome` would not be changed.
|
||||
* ``lookup(field if not empty, field if empty)`` -- like test, except the arguments are field (metadata) names, not text. The value of the appropriate field will be fetched and used. Note that because composite columns are fields, you can use this function in one composite field to use the value of some other composite field. This is extremely useful when constructing variable save paths (more later).
|
||||
* ``re(pattern, replacement)`` -- return the field after applying the regular expression. All instances of `pattern` are replaced with `replacement`. As in all of |app|, these are python-compatible regular expressions.
|
||||
@ -164,7 +165,7 @@ For tags, the result cut apart whereever |app| finds a comma. For example, if th
|
||||
|
||||
The same thing happens for authors, but using a different character for the cut, a `&` (ampersand) instead of a comma. For example, if the template produces the value ``Blogs, Joe&Posts, Susan``, then the book will end up with two authors, ``Blogs, Joe`` and ``Posts, Susan``. If the template produces the value ``Blogs, Joe;Posts, Susan``, then the book will have one author with a rather strange name.
|
||||
|
||||
Plugboards affect only the metadata written into the book. They do not affect calibre's metadata or the metadata used in ``save to disk`` and ``send to device`` templates. Plugboards also do not affect what is written into a Sony's database, so cannot be used for altering the metadata shown on a Sony's menu.
|
||||
Plugboards affect the metadata written into the book when it is saved to disk or written to the device. Plugboards do not affect the metadata used by ``save to disk`` and ``send to device`` to create the file names. Instead, file names are constructed using the templates entered on the appropriate preferences window.
|
||||
|
||||
Helpful Tips
|
||||
------------
|
||||
|
||||
@ -6,7 +6,7 @@ __docformat__ = 'restructuredtext en'
|
||||
Perform various initialization tasks.
|
||||
'''
|
||||
|
||||
import locale, sys, os
|
||||
import locale, sys, os, re
|
||||
|
||||
# Default translation is NOOP
|
||||
import __builtin__
|
||||
@ -114,6 +114,34 @@ if not _run_once:
|
||||
r, w, a, rb, wb, ab, r+, w+, a+, r+b, w+b, a+b
|
||||
'''
|
||||
if iswindows:
|
||||
class fwrapper(object):
|
||||
def __init__(self, name, fobject):
|
||||
object.__setattr__(self, 'fobject', fobject)
|
||||
object.__setattr__(self, 'name', name)
|
||||
|
||||
def __getattribute__(self, attr):
|
||||
if attr == 'name':
|
||||
return object.__getattribute__(self, attr)
|
||||
fobject = object.__getattribute__(self, 'fobject')
|
||||
return getattr(fobject, attr)
|
||||
|
||||
def __setattr__(self, attr, val):
|
||||
fobject = object.__getattribute__(self, 'fobject')
|
||||
return setattr(fobject, attr, val)
|
||||
|
||||
def __repr__(self):
|
||||
fobject = object.__getattribute__(self, 'fobject')
|
||||
name = object.__getattribute__(self, 'name')
|
||||
return re.sub(r'''['"]<fdopen>['"]''', repr(name),
|
||||
repr(fobject))
|
||||
|
||||
def __str__(self):
|
||||
return repr(self)
|
||||
|
||||
def __unicode__(self):
|
||||
return repr(self).decode('utf-8')
|
||||
|
||||
|
||||
m = mode[0]
|
||||
random = len(mode) > 1 and mode[1] == '+'
|
||||
binary = mode[-1] == 'b'
|
||||
@ -139,6 +167,7 @@ if not _run_once:
|
||||
flags |= os.O_NOINHERIT
|
||||
fd = os.open(name, flags)
|
||||
ans = os.fdopen(fd, mode, bufsize)
|
||||
ans = fwrapper(name, ans)
|
||||
else:
|
||||
import fcntl
|
||||
try:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -120,6 +120,9 @@ def utcfromtimestamp(stamp):
|
||||
|
||||
def format_date(dt, format, assume_utc=False, as_utc=False):
|
||||
''' Return a date formatted as a string using a subset of Qt's formatting codes '''
|
||||
if not format:
|
||||
format = 'dd MMM yyyy'
|
||||
|
||||
if hasattr(dt, 'tzinfo'):
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=_utc_tz if assume_utc else
|
||||
|
||||
@ -4,7 +4,9 @@ Created on 23 Sep 2010
|
||||
@author: charles
|
||||
'''
|
||||
|
||||
import re, string
|
||||
import re, string, traceback
|
||||
|
||||
from calibre.constants import DEBUG
|
||||
|
||||
class TemplateFormatter(string.Formatter):
|
||||
'''
|
||||
@ -19,7 +21,6 @@ class TemplateFormatter(string.Formatter):
|
||||
string.Formatter.__init__(self)
|
||||
self.book = None
|
||||
self.kwargs = None
|
||||
self.sanitize = None
|
||||
|
||||
def _lookup(self, val, field_if_set, field_not_set):
|
||||
if val:
|
||||
@ -39,6 +40,15 @@ class TemplateFormatter(string.Formatter):
|
||||
else:
|
||||
return value_if_not
|
||||
|
||||
def _switch(self, val, *args):
|
||||
i = 0
|
||||
while i < len(args):
|
||||
if i + 1 >= len(args):
|
||||
return args[i]
|
||||
if re.search(args[i], val):
|
||||
return args[i+1]
|
||||
i += 2
|
||||
|
||||
def _re(self, val, pattern, replacement):
|
||||
return re.sub(pattern, replacement, val)
|
||||
|
||||
@ -66,12 +76,19 @@ class TemplateFormatter(string.Formatter):
|
||||
'lookup' : (2, _lookup),
|
||||
're' : (2, _re),
|
||||
'shorten' : (3, _shorten),
|
||||
'switch' : (-1, _switch),
|
||||
'test' : (2, _test),
|
||||
}
|
||||
|
||||
format_string_re = re.compile(r'^(.*)\|(.*)\|(.*)$')
|
||||
compress_spaces = re.compile(r'\s+')
|
||||
|
||||
arg_parser = re.Scanner([
|
||||
(r',', lambda x,t: ''),
|
||||
(r'.*?((?<!\\),)', lambda x,t: t[:-1]),
|
||||
(r'.*?\)', lambda x,t: t[:-1]),
|
||||
])
|
||||
|
||||
def get_value(self, key, args, kwargs):
|
||||
raise Exception('get_value must be implemented in the subclass')
|
||||
|
||||
@ -83,8 +100,8 @@ class TemplateFormatter(string.Formatter):
|
||||
return fmt, '', ''
|
||||
return matches.groups()
|
||||
except:
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
return fmt, '', ''
|
||||
|
||||
def format_field(self, val, fmt):
|
||||
@ -105,7 +122,7 @@ class TemplateFormatter(string.Formatter):
|
||||
if fmt[colon:p] in self.functions:
|
||||
field = fmt[colon:p]
|
||||
func = self.functions[field]
|
||||
args = fmt[p+1:-1].split(',')
|
||||
args = self.arg_parser.scan(fmt[p+1:])[0]
|
||||
if (func[0] == 0 and (len(args) != 1 or args[0])) or \
|
||||
(func[0] > 0 and func[0] != len(args)):
|
||||
raise ValueError('Incorrect number of arguments for function '+ fmt[0:p])
|
||||
@ -123,14 +140,15 @@ class TemplateFormatter(string.Formatter):
|
||||
ans = string.Formatter.vformat(self, fmt, args, kwargs)
|
||||
return self.compress_spaces.sub(' ', ans).strip()
|
||||
|
||||
def safe_format(self, fmt, kwargs, error_value, book, sanitize=None):
|
||||
def safe_format(self, fmt, kwargs, error_value, book):
|
||||
self.kwargs = kwargs
|
||||
self.book = book
|
||||
self.sanitize = sanitize
|
||||
self.composite_values = {}
|
||||
try:
|
||||
ans = self.vformat(fmt, [], kwargs).strip()
|
||||
except:
|
||||
if DEBUG:
|
||||
traceback.print_exc()
|
||||
ans = error_value
|
||||
return ans
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -18,8 +18,9 @@ If this module is run, it will perform a series of unit tests.
|
||||
|
||||
import sys, string, operator
|
||||
|
||||
from calibre.utils.pyparsing import CaselessKeyword, Group, Forward, CharsNotIn, Suppress, \
|
||||
OneOrMore, MatchFirst, CaselessLiteral, Optional, NoMatch, ParseException
|
||||
from calibre.utils.pyparsing import CaselessKeyword, Group, Forward, \
|
||||
CharsNotIn, Suppress, OneOrMore, MatchFirst, CaselessLiteral, \
|
||||
Optional, NoMatch, ParseException, QuotedString
|
||||
from calibre.constants import preferred_encoding
|
||||
|
||||
|
||||
@ -127,18 +128,21 @@ class SearchQueryParser(object):
|
||||
location |= l
|
||||
location = Optional(location, default='all')
|
||||
word_query = CharsNotIn(string.whitespace + '()')
|
||||
quoted_query = Suppress('"')+CharsNotIn('"')+Suppress('"')
|
||||
#quoted_query = Suppress('"')+CharsNotIn('"')+Suppress('"')
|
||||
quoted_query = QuotedString('"', escChar='\\')
|
||||
query = quoted_query | word_query
|
||||
Token = Group(location + query).setResultsName('token')
|
||||
|
||||
if test:
|
||||
print 'Testing Token parser:'
|
||||
Token.validate()
|
||||
failed = SearchQueryParser.run_tests(Token, 'token',
|
||||
(
|
||||
('tag:asd', ['tag', 'asd']),
|
||||
('ddsä', ['all', 'ddsä']),
|
||||
('"one two"', ['all', 'one two']),
|
||||
('title:"one two"', ['title', 'one two']),
|
||||
(u'ddsä', ['all', u'ddsä']),
|
||||
('"one \\"two"', ['all', 'one "two']),
|
||||
('title:"one \\"1.5\\" two"', ['title', 'one "1.5" two']),
|
||||
('title:abc"def', ['title', 'abc"def']),
|
||||
)
|
||||
)
|
||||
|
||||
@ -167,7 +171,7 @@ class SearchQueryParser(object):
|
||||
).setResultsName("or") | And)
|
||||
|
||||
if test:
|
||||
Or.validate()
|
||||
#Or.validate()
|
||||
self._tests_failed = bool(failed)
|
||||
|
||||
self._parser = Or
|
||||
@ -240,6 +244,8 @@ class SearchQueryParser(object):
|
||||
'''
|
||||
return set([])
|
||||
|
||||
# Testing {{{
|
||||
|
||||
class Tester(SearchQueryParser):
|
||||
|
||||
texts = {
|
||||
@ -599,3 +605,6 @@ def main(args=sys.argv):
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
|
||||
# }}}
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user