KG 0.7.26 updates

This commit is contained in:
GRiker 2010-10-30 17:36:50 -07:00
commit 512da0d206
90 changed files with 36487 additions and 25774 deletions

View File

@ -4,6 +4,164 @@
# for important features/bug fixes.
# Also, each release can have new and improved recipes.
- version: 0.7.26
date: 2010-10-30
new features:
- title: "Check library: Allow wildcards in ignore names field"
bug fixes:
- title: "Fix regression in 0.7.25 that broke reading metadata from filenames."
- title: "Fix regression in 0.7.25 that caused original files to be mistakenly removed when adding books recursively"
- title: "Fix long series/publisher causing edit metadata in bulk dialog to become very large"
tickets: [7332]
- title: "Only add SONY periodical code to downloaded news if output profile is set to one of the SONY reader profiles. This is needed because the ever delightful Stanza crashes and burns when an EPUB has the periodical code"
improved recipes:
- El Periodico
- New Zealand Herald
new recipes:
- title: "Taggeschau.de"
author: "Florian Andreas Pfaff"
- title: "Gamespot Reviews"
author: "Marc Tonsing"
- version: 0.7.25
date: 2010-10-29
new features:
- title: "Add support for the SONY periodical format."
description: "This means that news downloaded by calibre and sent to a newer SONY device (350/650/900) should appear in the Periodicals section and have the special periodicals navigation user interface"
type: major
- title: "Content server: Make the new browsing interface the default. The old interface can be accessed at /old"
- title: "Content server: Allow running of content server as a WSGI application within another server. Add tutorial for this to the User Manual."
- title: "Support for the Pico Life reader, Kobo Wifi and HTC Aria"
- title: "Content server: Add a new --url-prefix command line option to ease the use of the server with a reverse proxy"
- title: "New social metadata plugin for Amazon that does not rely on AWS. Since Amazon broke AWS, it is recommended you upgrade to this version if you use metadata from Amazon"
- title: "Add a tweak to specify the fonts used when geenrating the default cover"
- title: "Add an output profile for generic Tablet devices"
tickets: [7289]
- title: "SONY driver: Allow sorting of collections by arbitrary field via a new tweak."
- title: "Content server: Make /mobile a little prettier"
- title: "Add button to 'Library Check' to automatically delete spurious files and folders"
bug fixes:
- title: "FB2 Input: Lots of love. Handle stylesheets and style attributes. Make parsinf malformed FB2 files more robust."
tickets: [7219, 7230]
- title: "Fix auto send of news to device with multiple calibre libraries. The fix means that if you have any pending news to be sent, it will be ignored after the update. Future news downloads will once again be automatically sent to the device."
- title: "MOBI Output: Conversion of super/sub scripts now handles nested tags."
tickets: [7264]
- title: "Conversion pipeline: Fix parsing of XML encoding declarations."
tickets: [7328]
- title: "Pandigital (Kobo): Upload thumbnails to correct location"
tickets: [7165]
- title: "Fix auto emailed news with non asci characters in title not being deliverd to Kindle"
tickets: [7322]
- title: "Read metadata only after on import plugins have run when adding books to GUI"
tickets: [7245]
- title: "Various fixes for bugs caused by non ascii temporary paths on windows with non UTF-8 filesystem encodings"
tickets: [7288]
- title: "Various fixes/enhancements to SNB Output"
- title: "Allow Tag editor in edit metadata dialog to be used even if tags have been changed"
tickets: [7298]
- title: "Fix crash on some OS X machines when Preferences->Conversion->Output is clicked"
- title: "MOBI indexing: Fix last entry missing sometimes"
tickets: [6595]
- title: "Fix regression causing books to be deselected after sending to device"
tickets: [7271]
- title: "Conversion pipeline: Fix rescaling of GIF images not working"
tickets: [7306]
- title: "Update PDF metadata/conversion libraries in windows build"
- title: "Fix timezone bug when searching on date fields"
tickets: [7300]
- title: "Fix regression that caused the viewr to crash if the main application is closed"
tickets: [7276]
- title: "Fix bug causing a spurious metadata.opf file to be written at the root of the calibre library when adding books"
- title: "Use the same title casing algorithm in all places"
- title: "Fix bulk edit of dual state boolean custom columns"
- title: "Increase image size for comics in Kindle DX profile for better conversion of comics to PDF"
- title: "Fix restore db to not dies when conflicting custom columns are encountered and report conflicting columns errors. Fix exceptions when referencing invalid _index fields."
- title: "Fix auto merge books not respecting article sort tweak"
tickets: [7147]
- title: "Linux device drivers: Fix udisks based ejecting for devices with multiple nodes"
- title: "Linux device mounting: Mount the drive with the lowest kernel name as main memory"
- title: "Fix use of numeric fields in templates"
- title: "EPUB Input: Handle EPUB files with multiple OPF files."
tickets: [7229]
- title: "Setting EPUB metadata: Fix date format. Fix language being overwritten by und when unspecified. Fix empty ISBN identifier being created"
- title: "Fix cannot delete a Series listing from List view also dismiss fetch metadata dialog when no metadata found automatically"
tickets: [7221, 7220]
- title: "Content server: Handle switch library in GUI gracefully"
- title: "calibre-server: Use cherrypy implementation of --pidfile and --daemonize"
new recipes:
- title: "Ming Pao"
author: "Eddie Lau"
- title: "lenta.ru"
author: "Nikolai Kotchetkov"
- title: "frazpc.pl"
author: "Tomasz Dlugosz"
- title: "Perfil and The Economic Collapse Blog"
author: "Darko Miletic"
- title: "STNN"
author: "Larry Chan"
improved recipes:
- CubaDebate
- El Pais
- Fox News
- New Scientist
- The Economic Times of India
- version: 0.7.24
date: 2010-10-17

View File

@ -8,24 +8,25 @@
<meta http-equiv="X-UA-Compatible" content="IE=100" />
<link rel="icon" type="image/x-icon" href="http://calibre-ebook.com/favicon.ico" />
<link rel="stylesheet" type="text/css" href="/static/browse/browse.css" />
<link type="text/css" href="/static/jquery_ui/css/humanity-custom/jquery-ui-1.8.5.custom.css" rel="stylesheet" />
<link rel="stylesheet" type="text/css" href="/static/jquery.multiselect.css" />
<link rel="stylesheet" type="text/css" href="{prefix}/static/browse/browse.css" />
<link type="text/css" href="{prefix}/static/jquery_ui/css/humanity-custom/jquery-ui-1.8.5.custom.css" rel="stylesheet" />
<link rel="stylesheet" type="text/css" href="{prefix}/static/jquery.multiselect.css" />
<script type="text/javascript" src="/static/jquery.js"></script>
<script type="text/javascript" src="/static/jquery.corner.js"></script>
<script type="text/javascript" src="{prefix}/static/jquery.js"></script>
<script type="text/javascript" src="{prefix}/static/jquery.corner.js"></script>
<script type="text/javascript"
src="/static/jquery_ui/js/jquery-ui-1.8.5.custom.min.js"></script>
src="{prefix}/static/jquery_ui/js/jquery-ui-1.8.5.custom.min.js"></script>
<script type="text/javascript"
src="/static/jquery.multiselect.min.js"></script>
src="{prefix}/static/jquery.multiselect.min.js"></script>
<script type="text/javascript" src="/static/browse/browse.js"></script>
<script type="text/javascript" src="{prefix}/static/browse/browse.js"></script>
<script type="text/javascript">
var sort_cookie_name = "{sort_cookie_name}";
var sort_select_label = "{sort_select_label}";
var url_prefix = "{prefix}";
$(document).ready(function() {{
init();
{script}
@ -39,16 +40,16 @@
<div id="header">
<div class="area">
<div class="bubble">
<p><a href="/browse" title="Return to top level"
<p><a href="{prefix}/browse" title="Return to top level"
>&rarr;&nbsp;home&nbsp;&larr;</a></p>
</div>
</div>
<div id="nav-container">&nbsp;
<ul id="primary-nav">
<li><a id="nav-mobile" href="/mobile" title="A version of this website suited for mobile browsers">Mobile</a></li>
<li><a id="nav-mobile" href="{prefix}/mobile" title="A version of this website suited for mobile browsers">Mobile</a></li>
<li><a id="nav-demo" href="/old" title="The old version of this webiste">Old</a></li>
<li><a id="nav-download" href="/opds" title="An OPDS feed based version of this website, used in special purpose applications">Feed</a></li>
<li><a id="nav-demo" href="{prefix}/old" title="The old version of this webiste">Old</a></li>
<li><a id="nav-download" href="{prefix}/opds" title="An OPDS feed based version of this website, used in special purpose applications">Feed</a></li>
</ul>
</div>
@ -58,7 +59,7 @@
<input type="hidden" name="cmd" value="_s-xclick"></input>
<input type="hidden" name="hosted_button_id" value="3028915"></input>
<input type="image"
src="http://calibre-ebook.com/site_media//img/button-donate.png"
src="{prefix}/static/button-donate.png"
name="submit"></input>
<img alt="" src="https://www.paypal.com/en_US/i/scr/pixel.gif"
width="1" height="1"></img>
@ -76,7 +77,7 @@
</select>
</div>
<div id="search_box">
<form name="search_form" action="/browse/search" method="get" accept-charset="UTF-8">
<form name="search_form" action="{prefix}/browse/search" method="get" accept-charset="UTF-8">
<input value="{initial_search}" type="text" title="Search" name="query"
class="search_input" />&nbsp;
<input type="submit" value="Search" title="Search" alt="Search" />

View File

@ -257,7 +257,7 @@ function booklist(hide_sort) {
function show_details(a_dom) {
var book = $(a_dom).closest('div.summary');
var bd = $('#book_details_dialog');
bd.html('<span class="loading"><img src="/static/loading.gif" alt="Loading" />Loading, please wait&hellip;</span>');
bd.html('<span class="loading"><img src="'+url_prefix+'/static/loading.gif" alt="Loading" />Loading, please wait&hellip;</span>');
bd.dialog('option', 'width', $(window).width() - 100);
bd.dialog('option', 'height', $(window).height() - 100);
bd.dialog('option', 'title', book.find('.title').text());

View File

@ -1,6 +1,6 @@
<div id="details_{id}" class="details">
<div class="left">
<img alt="Cover of {title}" src="/get/cover/{id}" />
<img alt="Cover of {title}" src="{prefix}/get/cover/{id}" />
</div>
<div class="right">
<div class="field formats">{formats}</div>

View File

@ -1,6 +1,6 @@
<div id="summary_{id}" class="summary">
<div class="left">
<img alt="Cover of {title}" src="/get/thumb_90_120/{id}" />
<img alt="Cover of {title}" src="{prefix}/get/thumb_90_120/{id}" />
{get_button}
</div>
<div class="right">
@ -8,7 +8,7 @@
<span class="rating_container">{stars}</span>
<span class="series">{series}</span>
<a href="#" onclick="show_details(this); return false;" title="{details_tt}">{details}</a>
<a href="/browse/book/{id}" title="{permalink_tt}">{permalink}</a>
<a href="{prefix}/browse/book/{id}" title="{permalink_tt}">{permalink}</a>
</div>
<div class="title"><strong>{title}</strong></div>
<div class="authors">{authors}</div>

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 KiB

View File

@ -40,7 +40,7 @@ function create_table_headers() {
function format_url(format, id, title) {
return 'get/'+format.toLowerCase() + '/'+encodeURIComponent(title) + '_' + id+'.'+format.toLowerCase();
return url_prefix + '/get/'+format.toLowerCase() + '/'+encodeURIComponent(title) + '_' + id+'.'+format.toLowerCase();
}
function render_book(book) {
@ -101,7 +101,7 @@ function render_book(book) {
}
}
title += '</span>'
title += '<img style="display:none" alt="" src="get/cover/{0}" /></span>'.format(id);
title += '<img style="display:none" alt="" src="{1}/get/cover/{0}" /></span>'.format(id, url_prefix);
title += '<div class="comments">{0}</div>'.format(comments)
// Render authors cell
var _authors = new Array();

View File

@ -3,26 +3,27 @@
<html xmlns="http://www.w3.org/1999/xhtml" version="XHTML 1.1" xml:lang="en">
<head>
<title>calibre library</title>
<link rel="stylesheet" type="text/css" href="/static/gui.css" charset="utf-8" />
<script type="text/javascript" src="/static/date.js" charset="utf-8"></script>
<script type="text/javascript" src="/static/jquery.js" charset="utf-8"></script>
<script type="text/javascript" src="/static/gui.js" charset="utf-8"></script>
<script type="text/javascript">var url_prefix='{prefix}';</script>
<link rel="stylesheet" type="text/css" href="{prefix}/static/gui.css" charset="utf-8" />
<script type="text/javascript" src="{prefix}/static/date.js" charset="utf-8"></script>
<script type="text/javascript" src="{prefix}/static/jquery.js" charset="utf-8"></script>
<script type="text/javascript" src="{prefix}/static/gui.js" charset="utf-8"></script>
<link rel="icon" href="http://calibre-ebook.com/favicon.ico" type="image/x-icon" />
</head>
<body>
<div id="banner">
<a style="border: 0pt" href="http://calibre-ebook.com" alt="calibre" title="calibre"><img style="border:0pt" src="/static/calibre_banner.png" alt="calibre" /></a>
<a style="border: 0pt" href="http://calibre-ebook.com" alt="calibre" title="calibre"><img style="border:0pt" src="{prefix}/static/calibre_banner.png" alt="calibre" /></a>
</div>
<div id="search_box">
<form name="search_form" onsubmit="search();return false;" action="./" method="get" accept-charset="UTF-8">
<form name="search_form" onsubmit="search();return false;" action="{prefix}/old" method="get" accept-charset="UTF-8">
<input value="" id="s" type="text" />
<input type="image" src="/static/btn_search_box.png" width="27" height="24" id="go" alt="Search" title="Search" />
<input type="image" src="{prefix}/static/btn_search_box.png" width="27" height="24" id="go" alt="Search" title="Search" />
</form>
</div>
<div id="count_bar">
<span id="left"><img src="/static/first.png" alt="Show first set of books" title="Show first set of books"/>&nbsp;<img src="/static/previous.png" alt="Show previous set of books" title="Show previous set of books"/>&nbsp;</span><span id="count">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span>&nbsp;<span id="right"><img src="/static/next.png" alt="Show next set of books" title="Show next set of books"/>&nbsp;<img src="/static/last.png" alt="Show last set of books" title="Show last set of books" /></span>
<span id="left"><img src="{prefix}/static/first.png" alt="Show first set of books" title="Show first set of books"/>&nbsp;<img src="{prefix}/static/previous.png" alt="Show previous set of books" title="Show previous set of books"/>&nbsp;</span><span id="count">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span>&nbsp;<span id="right"><img src="{prefix}/static/next.png" alt="Show next set of books" title="Show next set of books"/>&nbsp;<img src="{prefix}/static/last.png" alt="Show last set of books" title="Show last set of books" /></span>
</div>
<div id="main">
@ -38,7 +39,7 @@
<div id="loading">
<div>
<img align="top" src="/static/loading.gif" alt="Loading..." title="Loading..."/>&nbsp;<span id="loading_msg">Loading&hellip;</span>
<img align="top" src="{prefix}/static/loading.gif" alt="Loading..." title="Loading..."/>&nbsp;<span id="loading_msg">Loading&hellip;</span>
</div>
</div>

File diff suppressed because it is too large Load Diff

Binary file not shown.

After

Width:  |  Height:  |  Size: 781 B

View File

@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-
__license__ = 'GPL v3'
__copyright__ = '2009, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '30 October 2010, Jordi Balcells based on an earlier recipe by Darko Miletic <darko.miletic at gmail.com>'
'''
elperiodico.cat
'''
@ -12,8 +12,8 @@ from calibre.ebooks.BeautifulSoup import Tag
class ElPeriodico_cat(BasicNewsRecipe):
title = 'El Periodico de Catalunya'
__author__ = 'Darko Miletic'
description = 'Noticias desde Catalunya'
__author__ = 'Jordi Balcells/Darko Miletic'
description = 'Noticies des de Catalunya'
publisher = 'elperiodico.cat'
category = 'news, politics, Spain, Catalunya'
oldest_article = 2
@ -33,15 +33,25 @@ class ElPeriodico_cat(BasicNewsRecipe):
html2epub_options = 'publisher="' + publisher + '"\ncomments="' + description + '"\ntags="' + category + '"'
feeds = [(u"Tota l'edició", u'http://www.elperiodico.cat/rss.asp?id=46')]
feeds = [(u'Portada', u'http://www.elperiodico.cat/ca/rss/rss_portada.xml'),
(u'Internacional', u'http://www.elperiodico.cat/ca/rss/internacional/rss.xml'),
(u'Societat', u'http://www.elperiodico.cat/ca/rss/societat/rss.xml'),
(u'Ci\xe8ncia i tecnologia', u'http://www.elperiodico.cat/ca/rss/ciencia-i-tecnologia/rss.xml'),
(u'Esports', u'http://www.elperiodico.cat/ca/rss/esports/rss.xml'),
(u'Gent', u'http://www.elperiodico.cat/ca/rss/gent/rss.xml'),
(u'Opini\xf3', u'http://www.elperiodico.cat/ca/rss/opinio/rss.xml'),
(u'Pol\xedtica', u'http://www.elperiodico.cat/ca/rss/politica/rss.xml'),
(u'Barcelona', u'http://www.elperiodico.cat/ca/rss/barcelona/rss.xml'),
(u'Economia', u'http://www.elperiodico.cat/ca/rss/economia/rss.xml'),
(u'Cultura i espectacles', u'http://www.elperiodico.cat/ca/rss/cultura-i-espectacles/rss.xml'),
(u'Tele', u'http://www.elperiodico.cat/ca/rss/tele/rss.xml')]
keep_only_tags = [dict(name='div', attrs={'id':'noticia'})]
keep_only_tags = [dict(name='div', attrs={'class':'titularnoticia'}),
dict(name='div', attrs={'class':'noticia_completa'})]
remove_tags = [
dict(name=['object','link','script'])
,dict(name='ul',attrs={'class':'herramientasDeNoticia'})
,dict(name='div', attrs={'id':'inferiores'})
remove_tags = [dict(name='div', attrs={'class':['opcionb','opcionb last','columna_noticia']}),
dict(name='span', attrs={'class':'opcionesnoticia'})
]
def print_version(self, url):

View File

@ -2,17 +2,17 @@
# -*- coding: utf-8 -*-
__license__ = 'GPL v3'
__copyright__ = '2009, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '30 October 2010, Jordi Balcells based on an earlier recipe by Darko Miletic <darko.miletic at gmail.com>'
'''
elperiodico.com
elperiodico.cat
'''
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import Tag
class ElPeriodico_esp(BasicNewsRecipe):
class ElPeriodico_cat(BasicNewsRecipe):
title = 'El Periodico de Catalunya'
__author__ = 'Darko Miletic'
__author__ = 'Jordi Balcells/Darko Miletic'
description = 'Noticias desde Catalunya'
publisher = 'elperiodico.com'
category = 'news, politics, Spain, Catalunya'
@ -33,15 +33,25 @@ class ElPeriodico_esp(BasicNewsRecipe):
html2epub_options = 'publisher="' + publisher + '"\ncomments="' + description + '"\ntags="' + category + '"'
feeds = [(u"Toda la edición", u'http://www.elperiodico.com/rss.asp?id=46')]
feeds = [(u'Portada', u'http://www.elperiodico.com/es/rss/rss_portada.xml'),
(u'Internacional', u'http://elperiodico.com/es/rss/internacional/rss.xml'),
(u'Sociedad', u'http://elperiodico.com/es/rss/sociedad/rss.xml'),
(u'Ciencia y Tecnolog\xeda', u'http://elperiodico.com/es/rss/ciencia-y-tecnologia/rss.xml'),
(u'Deportes', u'http://elperiodico.com/es/rss/deportes/rss.xml'),
(u'Gente', u'http://elperiodico.com/es/rss/gente/rss.xml'),
(u'Opini\xf3n', u'http://elperiodico.com/es/rss/opinion/rss.xml'),
(u'Pol\xedtica', u'http://elperiodico.com/es/rss/politica/rss.xml'),
(u'Barcelona', u'http://elperiodico.com/es/rss/barcelona/rss.xml'),
(u'Econom\xeda', u'http://elperiodico.com/es/rss/economia/rss.xml'),
(u'Cultura y espect\xe1culos', u'http://elperiodico.com/es/rss/cultura-y-espectaculos/rss.xml'),
(u'Tele', u'http://elperiodico.com/es/rss/cultura-y-espectaculos/rss.xml')]
keep_only_tags = [dict(name='div', attrs={'id':'noticia'})]
keep_only_tags = [dict(name='div', attrs={'class':'titularnoticia'}),
dict(name='div', attrs={'class':'noticia_completa'})]
remove_tags = [
dict(name=['object','link','script'])
,dict(name='ul',attrs={'class':'herramientasDeNoticia'})
,dict(name='div', attrs={'id':'inferiores'})
remove_tags = [dict(name='div', attrs={'class':['opcionb','opcionb last','columna_noticia']}),
dict(name='span', attrs={'class':'opcionesnoticia'})
]
def print_version(self, url):

View File

@ -0,0 +1,41 @@
__license__ = 'GPL v3'
__author__ = u'Marc T\xf6nsing'
from calibre.web.feeds.news import BasicNewsRecipe
class GamespotCom(BasicNewsRecipe):
title = u'Gamespot.com Reviews'
description = 'review articles from gamespot.com'
language = 'en'
__author__ = u'Marc T\xf6nsing'
oldest_article = 7
max_articles_per_feed = 40
remove_empty_feeds = True
no_stylesheets = True
no_javascript = True
feeds = [
('PC Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=5'),
('XBOX 360 Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=1029'),
('Wii Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=1031'),
('PlayStation 3 Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=1028'),
('PlayStation 2 Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=7'),
('PlayStation Portable Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=1024'),
('Nintendo DS Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=1026'),
('iPhone Reviews', 'http://www.gamespot.com/rss/game_updates.php?type=5&platform=1049'),
]
remove_tags = [
dict(name='div', attrs={'class':'top_bar'}),
dict(name='div', attrs={'class':'video_embed'})
]
def get_cover_url(self):
return 'http://image.gamespotcdn.net/gamespot/shared/gs5/gslogo_bw.gif'
def get_article_url(self, article):
return article.get('link') + '?print=1'

View File

@ -0,0 +1,177 @@
#!/usr/bin/env python
'''
Lenta.ru
'''
from calibre.web.feeds.feedparser import parse
from calibre.ebooks.BeautifulSoup import Tag
from calibre.web.feeds.news import BasicNewsRecipe
import re
class LentaRURecipe(BasicNewsRecipe):
title = u'Lenta.ru: \u041d\u043e\u0432\u043e\u0441\u0442\u0438'
__author__ = 'Nikolai Kotchetkov'
publisher = 'lenta.ru'
category = 'news, Russia'
description = u'''\u0415\u0436\u0435\u0434\u043d\u0435\u0432\u043d\u0430\u044f
\u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0433\u0430\u0437\u0435\u0442\u0430.
\u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u0441\u043e
\u0432\u0441\u0435\u0433\u043e \u043c\u0438\u0440\u0430 \u043d\u0430
\u0440\u0443\u0441\u0441\u043a\u043e\u043c
\u044f\u0437\u044b\u043a\u0435'''
description = u'Ежедневная интернет-газета. Новости со всего мира на русском языке'
oldest_article = 3
max_articles_per_feed = 100
masthead_url = u'http://img.lenta.ru/i/logowrambler.gif'
cover_url = u'http://img.lenta.ru/i/logowrambler.gif'
#Add feed names if you want them to be sorted (feeds of this list appear first)
sortOrder = [u'_default', u'В России', u'б.СССР', u'В мире']
encoding = 'cp1251'
language = 'ru'
no_stylesheets = True
remove_javascript = True
recursions = 0
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
keep_only_tags = [dict(name='td', attrs={'class':['statya','content']})]
remove_tags_after = [dict(name='p', attrs={'class':'links'}), dict(name='div', attrs={'id':'readers-block'})]
remove_tags = [dict(name='table', attrs={'class':['vrezka','content']}), dict(name='div', attrs={'class':'b240'}), dict(name='div', attrs={'id':'readers-block'}), dict(name='p', attrs={'class':'links'})]
feeds = [u'http://lenta.ru/rss/']
extra_css = 'h1 {font-size: 1.2em; margin: 0em 0em 0em 0em;} h2 {font-size: 1.0em; margin: 0em 0em 0em 0em;} h3 {font-size: 0.8em; margin: 0em 0em 0em 0em;}'
def parse_index(self):
try:
feedData = parse(self.feeds[0])
if not feedData:
raise NotImplementedError
self.log("parse_index: Feed loaded successfully.")
if feedData.feed.has_key('title'):
self.title = feedData.feed.title
self.log("parse_index: Title updated to: ", self.title)
if feedData.feed.has_key('image'):
self.log("HAS IMAGE!!!!")
def get_virtual_feed_articles(feed):
if feeds.has_key(feed):
return feeds[feed][1]
self.log("Adding new feed: ", feed)
articles = []
feeds[feed] = (feed, articles)
return articles
feeds = {}
#Iterate feed items and distribute articles using tags
for item in feedData.entries:
link = item.get('link', '');
title = item.get('title', '');
if '' == link or '' == title:
continue
article = {'title':title, 'url':link, 'description':item.get('description', ''), 'date':item.get('date', ''), 'content':''};
if not item.has_key('tags'):
get_virtual_feed_articles('_default').append(article)
continue
for tag in item.tags:
addedToDefault = False
term = tag.get('term', '')
if '' == term:
if (not addedToDefault):
get_virtual_feed_articles('_default').append(article)
continue
get_virtual_feed_articles(term).append(article)
#Get feed list
#Select sorted feeds first of all
result = []
for feedName in self.sortOrder:
if (not feeds.has_key(feedName)): continue
result.append(feeds[feedName])
del feeds[feedName]
result = result + feeds.values()
return result
except Exception, err:
self.log(err)
raise NotImplementedError
def preprocess_html(self, soup):
return self.adeify_images(soup)
def postprocess_html(self, soup, first_fetch):
#self.log('Original: ', soup.prettify())
contents = Tag(soup, 'div')
#Extract tags with given attributes
extractElements = {'div' : [{'id' : 'readers-block'}]}
#Remove all elements that were not extracted before
for tag, attrs in extractElements.iteritems():
for attr in attrs:
garbage = soup.findAll(tag, attr)
if garbage:
for pieceOfGarbage in garbage:
pieceOfGarbage.extract()
#Find article text using header
#and add all elements to contents
element = soup.find({'h1' : True, 'h2' : True})
if (element):
element.name = 'h1'
while element:
nextElement = element.nextSibling
element.extract()
contents.insert(len(contents.contents), element)
element = nextElement
#Place article date after header
dates = soup.findAll(text=re.compile('\d{2}\.\d{2}\.\d{4}, \d{2}:\d{2}:\d{2}'))
if dates:
for date in dates:
for string in date:
parent = date.parent
if (parent and isinstance(parent, Tag) and 'div' == parent.name and 'dt' == parent['class']):
#Date div found
parent.extract()
parent['style'] = 'font-size: 0.5em; color: gray; font-family: monospace;'
contents.insert(1, parent)
break
#Place article picture after date
pic = soup.find('img')
if pic:
picDiv = Tag(soup, 'div')
picDiv['style'] = 'width: 100%; text-align: center;'
pic.extract()
picDiv.insert(0, pic)
title = pic.get('title', None)
if title:
titleDiv = Tag(soup, 'div')
titleDiv['style'] = 'font-size: 0.5em;'
titleDiv.insert(0, title)
picDiv.insert(1, titleDiv)
contents.insert(2, picDiv)
body = soup.find('td', {'class':['statya','content']})
if body:
body.replaceWith(contents)
#self.log('Result: ', soup.prettify())
return soup

View File

@ -1,53 +1,79 @@
#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2009, Mathieu Godlewski <mathieu at godlewski.fr>'
__copyright__ = '2009, Mathieu Godlewski <mathieu at godlewski.fr>; 2010, Louis Gesbert <meta at antislash dot info>'
'''
Mediapart
'''
import re, string
from calibre.ebooks.BeautifulSoup import BeautifulSoup
from calibre.ebooks.BeautifulSoup import Tag
from calibre.web.feeds.news import BasicNewsRecipe
class Mediapart(BasicNewsRecipe):
title = 'Mediapart'
__author__ = 'Mathieu Godlewski <mathieu at godlewski.fr>'
__author__ = 'Mathieu Godlewski'
description = 'Global news in french from online newspapers'
oldest_article = 7
language = 'fr'
needs_subscription = True
max_articles_per_feed = 50
no_stylesheets = True
html2lrf_options = ['--base-font-size', '10']
cover_url = 'http://www.mediapart.fr/sites/all/themes/mediapart/mediapart/images/annonce.jpg'
feeds = [
('Les articles', 'http://www.mediapart.fr/articles/feed'),
]
preprocess_regexps = [ (re.compile(i[0], re.IGNORECASE|re.DOTALL), i[1]) for i in
[
(r'<div class="print-title">([^>]+)</div>', lambda match : '<h2>'+match.group(1)+'</h2>'),
(r'<p>Mediapart\.fr</p>', lambda match : ''),
(r'<p[^>]*>[\s]*</p>', lambda match : ''),
(r'<p><a href="[^\.]+\.pdf">[^>]*</a></p>', lambda match : ''),
# -- print-version has poor quality on this website, better do the conversion ourselves
#
# preprocess_regexps = [ (re.compile(i[0], re.IGNORECASE|re.DOTALL), i[1]) for i in
# [
# (r'<div class="print-title">([^>]+)</div>', lambda match : '<h2>'+match.group(1)+'</h2>'),
# (r'<span class=\'auteur_staff\'>[^>]+<a title=\'[^\']*\'[^>]*>([^<]*)</a>[^<]*</span>',
# lambda match : '<i>'+match.group(1)+'</i>'),
# (r'\'', lambda match: '&rsquo;'),
# ]
# ]
#
# remove_tags = [ dict(name='div', attrs={'class':'print-source_url'}),
# dict(name='div', attrs={'class':'print-links'}),
# dict(name='img', attrs={'src':'entete_article.png'}),
# dict(name='br') ]
#
# def print_version(self, url):
# raw = self.browser.open(url).read()
# soup = BeautifulSoup(raw.decode('utf8', 'replace'))
# div = soup.find('div', {'id':re.compile('node-\d+')})
# if div is None:
# return None
# article_id = string.replace(div['id'], 'node-', '')
# if article_id is None:
# return None
# return 'http://www.mediapart.fr/print/'+article_id
# -- Non-print version [dict(name='div', attrs={'class':'advert'})]
keep_only_tags = [
dict(name='h1', attrs={'class':'title'}),
dict(name='div', attrs={'class':'page_papier_detail'}),
]
]
remove_tags = [ dict(name='div', attrs={'class':'print-source_url'}),
dict(name='div', attrs={'class':'print-links'}),
dict(name='img', attrs={'src':'entete_article.png'}),
]
def preprocess_html(self,soup):
for title in soup.findAll('div', {'class':'titre'}):
tag = Tag(soup, 'h3')
title.replaceWith(tag)
tag.insert(0,title)
return soup
# -- Handle login
def get_browser(self):
br = BasicNewsRecipe.get_browser()
if self.username is not None and self.password is not None:
br.open('http://www.mediapart.fr/')
br.select_form(nr=1)
br['name'] = self.username
br['pass'] = self.password
br.submit()
return br
def print_version(self, url):
raw = self.browser.open(url).read()
soup = BeautifulSoup(raw.decode('utf8', 'replace'))
div = soup.find('div', {'class':'node node-type-article'})
if div is None:
return None
article_id = string.replace(div['id'], 'node-', '')
if article_id is None:
return None
return 'http://www.mediapart.fr/print/'+article_id

View File

@ -1,74 +1,43 @@
from calibre.web.feeds.recipes import BasicNewsRecipe
import re
class NewZealandHerald(BasicNewsRecipe):
title = 'New Zealand Herald'
__author__ = 'Krittika Goyal'
__author__ = 'Kovid Goyal'
description = 'Daily news'
timefmt = ' [%d %b, %Y]'
language = 'en_NZ'
oldest_article = 2.5
no_stylesheets = True
remove_tags_before = dict(name='div', attrs={'class':'contentContainer left eight'})
remove_tags_after = dict(name='div', attrs={'class':'callToAction'})
remove_tags = [
dict(name='iframe'),
dict(name='div', attrs={'class':['sectionHeader', 'tools','callToAction', 'contentContainer right two nopad relatedColumn']}),
#dict(name='div', attrs={'id':['shareContainer']}),
#dict(name='form', attrs={'onsubmit':"return verifySearch(this.w,'Keyword, citation, or #author')"}),
#dict(name='table', attrs={'cellspacing':'0'}),
feeds = [
('Business',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000003.xml'),
('World',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000002.xml'),
('National',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000001.xml'),
('Entertainment',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_001501119.xml'),
('Travel',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000007.xml'),
('Opinion',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000466.xml'),
('Life & Style',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000006.xml'),
('Technology'
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000005.xml'),
('Sport',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000004.xml'),
('Motoring',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000009.xml'),
('Property',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000008.xml'),
]
def preprocess_html(self, soup):
table = soup.find('table')
if table is not None:
table.extract()
return soup
#TO GET ARTICLES IN SECTION
def nz_parse_section(self, url):
soup = self.index_to_soup(url)
div = soup.find(attrs={'class':'col-300 categoryList'})
date = div.find(attrs={'class':'link-list-heading'})
current_articles = []
for x in date.findAllNext(attrs={'class':['linkList', 'link-list-heading']}):
if x.get('class') == 'link-list-heading': break
for li in x.findAll('li'):
a = li.find('a', href=True)
if a is None:
continue
title = self.tag_to_string(a)
url = a.get('href', False)
if not url or not title:
continue
if url.startswith('/'):
url = 'http://www.nzherald.co.nz'+url
self.log('\t\tFound article:', title)
self.log('\t\t\t', url)
current_articles.append({'title': title, 'url':url,
'description':'', 'date':''})
return current_articles
# To GET SECTIONS
def parse_index(self):
feeds = []
for title, url in [
('National',
'http://www.nzherald.co.nz/nz/news/headlines.cfm?c_id=1'),
('World',
'http://www.nzherald.co.nz/world/news/headlines.cfm?c_id=2'),
('Politics',
'http://www.nzherald.co.nz/politics/news/headlines.cfm?c_id=280'),
('Crime',
'http://www.nzherald.co.nz/crime/news/headlines.cfm?c_id=30'),
('Environment',
'http://www.nzherald.co.nz/environment/news/headlines.cfm?c_id=39'),
]:
articles = self.nz_parse_section(url)
if articles:
feeds.append((title, articles))
return feeds
def print_version(self, url):
m = re.search(r'objectid=(\d+)', url)
if m is None:
return url
return 'http://www.nzherald.co.nz/news/print.cfm?pnum=1&objectid=' + m.group(1)

View File

@ -0,0 +1,66 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
'''
perfil.com
'''
from calibre.web.feeds.news import BasicNewsRecipe
class Perfil(BasicNewsRecipe):
title = 'Perfil'
__author__ = 'Darko Miletic'
description = 'Noticias de Argentina y el resto del mundo'
publisher = 'perfil.com'
category = 'news, politics, Argentina'
oldest_article = 2
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'cp1252'
use_embedded_content = False
language = 'es'
remove_empty_feeds = True
masthead_url = 'http://www.perfil.com/export/sites/diarioperfil/arte/10/logo_perfilcom_mm.gif'
extra_css = """
body{font-family: Arial,Helvetica,sans-serif }
.seccion{border-bottom: 1px dotted #666666; text-transform: uppercase; font-size: x-large}
.foto1 h1{font-size: x-small}
h1{font-family: Georgia,"Times New Roman",serif}
img{margin-bottom: 0.4em}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
remove_tags = [
dict(name=['iframe','embed','object','base','meta','link'])
,dict(name='a', attrs={'href':'#comentarios'})
,dict(name='div', attrs={'class':'foto3'})
,dict(name='img', attrs={'alt':'ampliar'})
]
keep_only_tags=[dict(attrs={'class':['bd468a','cuerpoSuperior']})]
remove_attributes=['onload','lang','width','height','border']
feeds = [
(u'Ultimo momento' , u'http://www.perfil.com/rss/ultimomomento.xml')
,(u'Politica' , u'http://www.perfil.com/rss/politica.xml' )
,(u'Policia' , u'http://www.perfil.com/rss/policia.xml' )
,(u'Internacionales', u'http://www.perfil.com/rss/internacional.xml')
,(u'Economia' , u'http://www.perfil.com/rss/economia.xml' )
,(u'Deportes' , u'http://www.perfil.com/rss/deportes.xml' )
,(u'Opinion' , u'http://www.perfil.com/rss/columnistas.xml' )
,(u'Sociedad' , u'http://www.perfil.com/rss/sociedad.xml' )
,(u'Cultura' , u'http://www.perfil.com/rss/cultura.xml' )
,(u'Espectaculos' , u'http://www.perfil.com/rss/espectaculos.xml' )
,(u'Ciencia' , u'http://www.perfil.com/rss/ciencia.xml' )
,(u'Salud' , u'http://www.perfil.com/rss/salud.xml' )
,(u'Tecnologia' , u'http://www.perfil.com/rss/tecnologia.xml' )
]
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
return soup

View File

@ -0,0 +1,53 @@
__license__ = 'GPL v3'
__copyright__ = '2010, Louis Gesbert <meta at antislash dot info>'
'''
Rue89
'''
__author__ = '2010, Louis Gesbert <meta at antislash dot info>'
import re
from calibre.ebooks.BeautifulSoup import Tag
from calibre.web.feeds.news import BasicNewsRecipe
class Rue89(BasicNewsRecipe):
title = 'Rue89'
__author__ = 'Louis Gesbert'
description = 'Popular free french news website'
title = u'Rue89'
language = 'fr'
oldest_article = 7
max_articles_per_feed = 50
feeds = [(u'La Une', u'http://www.rue89.com/homepage/feed')]
no_stylesheets = True
preprocess_regexps = [
(re.compile(r'<(/?)h2>', re.IGNORECASE|re.DOTALL),
lambda match : '<'+match.group(1)+'h3>'),
(re.compile(r'<div class="print-title">([^>]+)</div>', re.IGNORECASE|re.DOTALL),
lambda match : '<h2>'+match.group(1)+'</h2>'),
(re.compile(r'<img[^>]+src="[^"]*/numeros/(\d+)[^0-9.">]*.gif"[^>]*/>', re.IGNORECASE|re.DOTALL),
lambda match : '<span style="font-family: Sans-serif; color: red; font-size:24pt; padding=2pt;">'+match.group(1)+'</span>'),
(re.compile(r'\''), lambda match: '&rsquo;'),
]
def preprocess_html(self,soup):
body = Tag(soup, 'body')
title = soup.find('h1', {'class':'title'})
content = soup.find('div', {'class':'content'})
soup.body.replaceWith(body)
body.insert(0, title)
body.insert(1, content)
return soup
remove_tags = [ #dict(name='div', attrs={'class':'print-source_url'}),
#dict(name='div', attrs={'class':'print-links'}),
#dict(name='img', attrs={'class':'print-logo'}),
dict(name='div', attrs={'class':'content_top'}),
dict(name='div', attrs={'id':'sidebar-left'}), ]
# -- print-version has poor quality on this website, better do the conversion ourselves
# def print_version(self, url):
# return re.sub('^.*-([0-9]+)$', 'http://www.rue89.com/print/\\1',url)

View File

@ -0,0 +1,24 @@
from calibre.web.feeds.news import BasicNewsRecipe
class Tagesschau(BasicNewsRecipe):
title = 'Tagesschau'
description = 'Nachrichten der ARD'
publisher = 'ARD'
language = 'de_DE'
__author__ = 'Florian Andreas Pfaff'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
feeds = [('Tagesschau', 'http://www.tagesschau.de/xml/rss2')]
remove_tags = [
dict(name='div', attrs={'class':['linksZumThema schmal','teaserBox','boxMoreLinks','directLinks','teaserBox boxtext','fPlayer','zitatBox breit flashaudio']}),
dict(name='div',
attrs={'id':['socialBookmarks','seitenanfang']}),
dict(name='ul',
attrs={'class':['directLinks','directLinks weltatlas']}),
dict(name='strong', attrs={'class':['boxTitle inv','inv']})
]
keep_only_tags = [dict(name='div', attrs={'id':'centerCol'})]

View File

@ -30,23 +30,40 @@
<title>
<xsl:value-of select="fb:description/fb:title-info/fb:book-title"/>
</title>
<style type="text/x-oeb1-css">
A { color : #0002CC }
A:HOVER { color : #BF0000 }
BODY { background-color : #FEFEFE; color : #000000; font-family : Verdana, Geneva, Arial, Helvetica, sans-serif; text-align : justify }
H1{ font-size : 160%; font-style : normal; font-weight : bold; text-align : left; border : 1px solid Black; background-color : #E7E7E7; margin-left : 0px; page-break-before : always; }
H2{ font-size : 130%; font-style : normal; font-weight : bold; text-align : left; background-color : #EEEEEE; border : 1px solid Gray; page-break-before : always; }
H3{ font-size : 110%; font-style : normal; font-weight : bold; text-align : left; background-color : #F1F1F1; border : 1px solid Silver;}
H4{ font-size : 100%; font-style : normal; font-weight : bold; text-align : left; border : 1px solid Gray; background-color : #F4F4F4;}
H5{ font-size : 100%; font-style : italic; font-weight : bold; text-align : left; border : 1px solid Gray; background-color : #F4F4F4;}
H6{ font-size : 100%; font-style : italic; font-weight : normal; text-align : left; border : 1px solid Gray; background-color : #F4F4F4;}
SMALL{ font-size : 80% }
BLOCKQUOTE{ margin-left :4em; margin-top:1em; margin-right:0.2em;}
HR{ color : Black }
DIV{font-family : "Times New Roman", Times, serif; text-align : justify}
UL{margin-left: 0}
.epigraph{width:50%; margin-left : 35%;}
<style type="text/css">
a { color : #0002CC }
a:hover { color : #BF0000 }
body { background-color : #FEFEFE; color : #000000; font-family : Verdana, Geneva, Arial, Helvetica, sans-serif; text-align : justify }
h1{ font-size : 160%; font-style : normal; font-weight : bold; text-align : left; border : 1px solid Black; background-color : #E7E7E7; margin-left : 0px; page-break-before : always; }
h2{ font-size : 130%; font-style : normal; font-weight : bold; text-align : left; background-color : #EEEEEE; border : 1px solid Gray; page-break-before : always; }
h3{ font-size : 110%; font-style : normal; font-weight : bold; text-align : left; background-color : #F1F1F1; border : 1px solid Silver;}
h4{ font-size : 100%; font-style : normal; font-weight : bold; text-align : left; border : 1px solid Gray; background-color : #F4F4F4;}
h5{ font-size : 100%; font-style : italic; font-weight : bold; text-align : left; border : 1px solid Gray; background-color : #F4F4F4;}
h6{ font-size : 100%; font-style : italic; font-weight : normal; text-align : left; border : 1px solid Gray; background-color : #F4F4F4;}
small { font-size : 80% }
blockquote { margin-left :4em; margin-top:1em; margin-right:0.2em;}
hr { color : Black }
div {font-family : "Times New Roman", Times, serif; text-align : justify}
ul {margin-left: 0}
.epigraph{width:50%; margin-left : 35%;}
div.paragraph { text-align: justify; text-indent: 2em; }
</style>
<link rel="stylesheet" type="text/css" href="inline-styles.css" />
</head>
<body>
<xsl:for-each select="fb:description/fb:title-info/fb:annotation">
@ -136,12 +153,13 @@
</xsl:choose>
</xsl:variable>
<xsl:if test="$section_has_title = 'None'">
<a name="TOC_{generate-id()}" />
<xsl:if test="@id">
<xsl:element name="a">
<xsl:attribute name="name"><xsl:value-of select="@id"/></xsl:attribute>
</xsl:element>
</xsl:if>
<div id="TOC_{generate-id()}">
<xsl:if test="@id">
<xsl:element name="a">
<xsl:attribute name="id"><xsl:value-of select="@id"/></xsl:attribute>
</xsl:element>
</xsl:if>
</div>
</xsl:if>
<xsl:apply-templates>
<xsl:with-param name="section_toc_id" select="$section_has_title" />
@ -160,13 +178,13 @@
</xsl:if>
<xsl:if test="$section_toc_id != 'None'">
<xsl:element name="a">
<xsl:attribute name="name">TOC_<xsl:value-of select="$section_toc_id"/></xsl:attribute>
<xsl:attribute name="id">TOC_<xsl:value-of select="$section_toc_id"/></xsl:attribute>
</xsl:element>
</xsl:if>
<a name="TOC_{generate-id()}"></a>
<xsl:if test="@id">
<xsl:element name="a">
<xsl:attribute name="name"><xsl:value-of select="@id"/></xsl:attribute>
<xsl:attribute name="id"><xsl:value-of select="@id"/></xsl:attribute>
</xsl:element>
</xsl:if>
<xsl:apply-templates/>
@ -176,7 +194,7 @@
<xsl:element name="h6">
<xsl:if test="@id">
<xsl:element name="a">
<xsl:attribute name="name"><xsl:value-of select="@id"/></xsl:attribute>
<xsl:attribute name="id"><xsl:value-of select="@id"/></xsl:attribute>
</xsl:element>
</xsl:if>
<xsl:apply-templates/>
@ -207,11 +225,18 @@
</xsl:template>
<!-- p -->
<xsl:template match="fb:p">
<div align="justify"><xsl:if test="@id">
<xsl:element name="div">
<xsl:attribute name="class">paragraph</xsl:attribute>
<xsl:if test="@id">
<xsl:element name="a">
<xsl:attribute name="name"><xsl:value-of select="@id"/></xsl:attribute>
</xsl:element>
</xsl:if> &#160;&#160;&#160;<xsl:apply-templates/></div>
</xsl:if>
<xsl:if test="@style">
<xsl:attribute name="style"><xsl:value-of select="@style"/></xsl:attribute>
</xsl:if>
<xsl:apply-templates/>
</xsl:element>
</xsl:template>
<!-- strong -->
<xsl:template match="fb:strong">

View File

@ -20,20 +20,4 @@ function setup_image_scaling_handlers() {
});
}
function extract_svged_images() {
$("svg").each(function() {
var children = $(this).children("img");
if (children.length == 1) {
var img = $(children[0]);
var href = img.attr('xlink:href');
if (href != undefined) {
$(this).replaceWith('<div style="text-align:center; margin: 0; padding: 0"><img style="height: 98%" alt="SVG Image" src="' + href +'"></img></div>');
}
}
});
}
$(document).ready(function() {
//extract_svged_images();
});

View File

@ -8,7 +8,7 @@ __docformat__ = 'restructuredtext en'
__all__ = [
'pot', 'translations', 'get_translations', 'iso639',
'build', 'build_pdf2xml',
'build', 'build_pdf2xml', 'server',
'gui',
'develop', 'install',
'resources',
@ -35,6 +35,9 @@ from setup.extensions import Build, BuildPDF2XML
build = Build()
build_pdf2xml = BuildPDF2XML()
from setup.server import Server
server = Server()
from setup.install import Develop, Install, Sdist
develop = Develop()
install = Install()

48
setup/server.py Normal file
View File

@ -0,0 +1,48 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
__license__ = 'GPL v3'
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import subprocess, tempfile, os, time
from setup import Command
class Server(Command):
description = 'Run the calibre server in development mode conveniently'
MONOCLE_PATH = '../monocle'
def rebuild_monocole(self):
subprocess.check_call(['sprocketize', '-C', self.MONOCLE_PATH,
'-I', 'src', 'src/monocle.js'],
stdout=open('resources/content_server/monocle.js', 'wb'))
def launch_server(self, log):
self.rebuild_monocole()
p = subprocess.Popen(['calibre-server', '--develop'],
stderr=subprocess.STDOUT, stdout=log)
return p
def run(self, opts):
tdir = tempfile.gettempdir()
logf = os.path.join(tdir, 'calibre-server.log')
log = open(logf, 'ab')
print 'Server log available at:', logf
while True:
print 'Starting server...'
p = self.launch_server(log)
try:
raw_input('Press Enter to kill/restart server. Ctrl+C to quit: ')
except:
break
else:
while p.returncode is None:
p.terminate()
time.sleep(0.1)
p.kill()
print

View File

@ -2,7 +2,7 @@ __license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'
__appname__ = 'calibre'
__version__ = '0.7.24'
__version__ = '0.7.26'
__author__ = "Kovid Goyal <kovid@kovidgoyal.net>"
import re

View File

@ -292,7 +292,7 @@ class RTFMetadataReader(MetadataReaderPlugin):
def get_metadata(self, stream, ftype):
from calibre.ebooks.metadata.rtf import get_metadata
return get_metadata(stream)
class SNBMetadataReader(MetadataReaderPlugin):
name = 'Read SNB metadata'
@ -471,7 +471,8 @@ from calibre.devices.iriver.driver import IRIVER_STORY
from calibre.devices.binatone.driver import README
from calibre.devices.hanvon.driver import N516, EB511, ALEX, AZBOOKA, THEBOOK
from calibre.devices.edge.driver import EDGE
from calibre.devices.teclast.driver import TECLAST_K3, NEWSMY, IPAPYRUS, SOVOS
from calibre.devices.teclast.driver import TECLAST_K3, NEWSMY, IPAPYRUS, \
SOVOS, PICO
from calibre.devices.sne.driver import SNE
from calibre.devices.misc import PALMPRE, AVANT, SWEEX, PDNOVEL, KOGAN, \
GEMEI, VELOCITYMICRO, PDNOVEL_KOBO
@ -572,6 +573,7 @@ plugins += [
ELONEX,
TECLAST_K3,
NEWSMY,
PICO,
IPAPYRUS,
SOVOS,
EDGE,

View File

@ -259,6 +259,9 @@ class OutputProfile(Plugin):
#: Number of ems that the left margin of a blockquote is rendered as
mobi_ems_per_blockquote = 1.0
#: Special periodical formatting needed in EPUB
epub_periodical_format = None
@classmethod
def tags_to_string(cls, tags):
return escape(', '.join(tags))
@ -439,6 +442,9 @@ class SonyReaderOutput(OutputProfile):
fsizes = [7.5, 9, 10, 12, 15.5, 20, 22, 24]
unsupported_unicode_chars = [u'\u201f', u'\u201b']
epub_periodical_format = 'sony'
#periodical_date_in_title = False
class KoboReaderOutput(OutputProfile):
@ -561,6 +567,8 @@ class CybookOpusOutput(SonyReaderOutput):
fbase = 16
fsizes = [12, 12, 14, 16, 18, 20, 22, 24]
epub_periodical_format = None
class KindleOutput(OutputProfile):
name = 'Kindle'
@ -658,13 +666,14 @@ class NookOutput(OutputProfile):
class BambookOutput(OutputProfile):
author = 'Li Fanxi'
name = 'Sanda Bambook'
short_name = 'bambook'
description = _('This profile is intended for the Sanda Bambook.')
# Screen size is a best guess
screen_size = (800, 600)
comic_screen_size = (700, 540)
screen_size = (600, 800)
comic_screen_size = (540, 700)
dpi = 168.451
fbase = 12
fsizes = [10, 12, 14, 16]

View File

@ -117,6 +117,12 @@ class PDNOVEL_KOBO(PDNOVEL):
EBOOK_DIR_MAIN = 'eBooks/Kobo'
def upload_cover(self, path, filename, metadata, filepath):
coverdata = getattr(metadata, 'thumbnail', None)
if coverdata and coverdata[2]:
with open(os.path.join(path, '.thumbnail', filename+'.jpg'), 'wb') as coverfile:
coverfile.write(coverdata[2])
class VELOCITYMICRO(USBMS):
name = 'VelocityMicro device interface'

View File

@ -41,6 +41,15 @@ class NEWSMY(TECLAST_K3):
WINDOWS_MAIN_MEM = 'NEWSMY'
WINDOWS_CARD_A_MEM = 'USBDISK____SD'
class PICO(NEWSMY):
name = 'Pico device interface'
gui_name = 'Pico'
description = _('Communicate with the Pico reader.')
WINDOWS_MAIN_MEM = 'USBDISK__USER'
EBOOK_DIR_MAIN = 'Books'
FORMATS = ['EPUB', 'FB2', 'TXT', 'LRC', 'PDB', 'PDF', 'HTML', 'WTXT']
class IPAPYRUS(TECLAST_K3):
name = 'iPapyrus device interface'

View File

@ -30,9 +30,9 @@ def detect(aBuf):
# Added by Kovid
ENCODING_PATS = [
re.compile(r'<\?[^<>]+encoding=[\'"](.*?)[\'"][^<>]*>',
re.compile(r'<\?[^<>]+encoding\s*=\s*[\'"](.*?)[\'"][^<>]*>',
re.IGNORECASE),
re.compile(r'''<meta\s+?[^<>]+?content=['"][^'"]*?charset=([-a-z0-9]+)[^'"]*?['"][^<>]*>''',
re.compile(r'''<meta\s+?[^<>]+?content\s*=\s*['"][^'"]*?charset=([-a-z0-9]+)[^'"]*?['"][^<>]*>''',
re.IGNORECASE)
]
ENTITY_PATTERN = re.compile(r'&(\S+?);')

View File

@ -187,9 +187,10 @@ class EPUBOutput(OutputFormatPlugin):
metadata_xml = None
extra_entries = []
if self.is_periodical:
from calibre.ebooks.epub.periodical import sony_metadata
metadata_xml, atom_xml = sony_metadata(oeb)
extra_entries = [('atom.xml', 'application/atom+xml', atom_xml)]
if self.opts.output_profile.epub_periodical_format == 'sony':
from calibre.ebooks.epub.periodical import sony_metadata
metadata_xml, atom_xml = sony_metadata(oeb)
extra_entries = [('atom.xml', 'application/atom+xml', atom_xml)]
oeb_output = plugin_for_output_format('oeb')
oeb_output.convert(oeb, tdir, input_plugin, opts, log)
opf = [x for x in os.listdir(tdir) if x.endswith('.opf')][0]

View File

@ -40,14 +40,35 @@ class FB2Input(InputFormatPlugin):
accelerators):
from calibre.ebooks.metadata.opf2 import OPFCreator
from calibre.ebooks.metadata.meta import get_metadata
from calibre.ebooks.oeb.base import XLINK_NS
from calibre.ebooks.oeb.base import XLINK_NS, XHTML_NS, RECOVER_PARSER
NAMESPACES = {'f':FB2NS, 'l':XLINK_NS}
log.debug('Parsing XML...')
raw = stream.read()
raw = stream.read().replace('\0', '')
try:
doc = etree.fromstring(raw)
except etree.XMLSyntaxError:
doc = etree.fromstring(raw.replace('& ', '&amp;'))
try:
doc = etree.fromstring(raw, parser=RECOVER_PARSER)
except:
doc = etree.fromstring(raw.replace('& ', '&amp;'),
parser=RECOVER_PARSER)
stylesheets = doc.xpath('//*[local-name() = "stylesheet" and @type="text/css"]')
css = ''
for s in stylesheets:
css += etree.tostring(s, encoding=unicode, method='text',
with_tail=False) + '\n\n'
if css:
import cssutils, logging
parser = cssutils.CSSParser(fetcher=None,
log=logging.getLogger('calibre.css'))
XHTML_CSS_NAMESPACE = '@namespace "%s";\n' % XHTML_NS
text = XHTML_CSS_NAMESPACE + css
log.debug('Parsing stylesheet...')
stylesheet = parser.parseString(text)
stylesheet.namespaces['h'] = XHTML_NS
css = unicode(stylesheet.cssText).replace('h|style', 'h|span')
css = re.sub(r'name\s*=\s*', 'class=', css)
self.extract_embedded_content(doc)
log.debug('Converting XML to HTML...')
ss = open(P('templates/fb2.xsl'), 'rb').read()
@ -63,7 +84,9 @@ class FB2Input(InputFormatPlugin):
for img in result.xpath('//img[@src]'):
src = img.get('src')
img.set('src', self.binary_map.get(src, src))
open('index.xhtml', 'wb').write(transform.tostring(result))
index = transform.tostring(result)
open('index.xhtml', 'wb').write(index)
open('inline-styles.css', 'wb').write(css)
stream.seek(0)
mi = get_metadata(stream, 'fb2')
if not mi.title:

View File

@ -282,15 +282,22 @@ class HTMLInput(InputFormatPlugin):
basedir = os.getcwd()
self.opts = opts
fname = None
if hasattr(stream, 'name'):
basedir = os.path.dirname(stream.name)
fname = os.path.basename(stream.name)
if file_ext != 'opf':
if opts.dont_package:
raise ValueError('The --dont-package option is not supported for an HTML input file')
from calibre.ebooks.metadata.html import get_metadata
oeb = self.create_oebbook(stream.name, basedir, opts, log,
get_metadata(stream))
mi = get_metadata(stream)
if fname:
from calibre.ebooks.metadata.meta import metadata_from_filename
fmi = metadata_from_filename(fname)
fmi.smart_update(mi)
mi = fmi
oeb = self.create_oebbook(stream.name, basedir, opts, log, mi)
return oeb
from calibre.ebooks.conversion.plumber import create_oebbook

View File

@ -8,88 +8,118 @@ Fetch metadata using Amazon AWS
'''
import sys, re
from lxml import etree
from lxml import html
from calibre import browser
from calibre.utils.date import parse_date, utcnow
from calibre.ebooks.metadata import MetaInformation, string_to_authors
from calibre.ebooks.metadata import check_isbn
from calibre.ebooks.metadata.book.base import Metadata
from calibre.ebooks.chardet import xml_to_unicode
AWS_NS = 'http://webservices.amazon.com/AWSECommerceService/2005-10-05'
def find_asin(br, isbn):
q = 'http://www.amazon.com/s?field-keywords='+isbn
raw = br.open_novisit(q).read()
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
root = html.fromstring(raw)
revs = root.xpath('//*[@class="asinReviewsSummary" and @name]')
revs = [x.get('name') for x in revs]
if revs:
return revs[0]
def AWS(tag):
return '{%s}%s'%(AWS_NS, tag)
class ISBNNotFound(ValueError):
pass
def check_for_errors(root, isbn):
err = root.find('.//'+AWS('Error'))
if err is not None:
text = etree.tostring(err, method='text', pretty_print=True,
encoding=unicode)
if 'AWS.InvalidParameterValue'+isbn in text:
raise ISBNNotFound(isbn)
raise Exception('Failed to get metadata with error: '\
+ text)
def to_asin(br, isbn):
if len(isbn) == 13:
try:
asin = find_asin(br, isbn)
except:
import traceback
traceback.print_exc()
asin = None
else:
asin = isbn
return asin
def get_social_metadata(title, authors, publisher, isbn):
mi = MetaInformation(title, authors)
if isbn:
br = browser()
response_xml = br.open('http://status.calibre-ebook.com/aws/metadata/'+isbn).read()
root = etree.fromstring(response_xml)
try:
check_for_errors(root, isbn)
except ISBNNotFound:
return mi
mi.title = root.findtext('.//'+AWS('Title'))
authors = [x.text for x in root.findall('.//'+AWS('Author'))]
if authors:
mi.authors = []
for x in authors:
mi.authors.extend(string_to_authors(x))
mi.publisher = root.findtext('.//'+AWS('Publisher'))
try:
d = root.findtext('.//'+AWS('PublicationDate'))
if d:
default = utcnow().replace(day=15)
d = parse_date(d[0].text, assume_utc=True, default=default)
mi.pubdate = d
except:
pass
try:
rating = float(root.findtext('.//'+AWS('AverageRating')))
num_of_reviews = int(root.findtext('.//'+AWS('TotalReviews')))
if num_of_reviews > 4 and rating > 0 and rating < 5:
mi.rating = rating
except:
pass
tags = [x.text for x in root.findall('.//%s/%s'%(AWS('Subjects'),
AWS('Subject')))]
if tags:
mi.tags = []
for x in tags:
mi.tags.extend([y.strip() for y in x.split('/')])
mi.tags = [x.replace(',', ';') for x in mi.tags]
comments = root.find('.//%s/%s'%(AWS('EditorialReview'),
AWS('Content')))
if comments is not None:
mi.comments = etree.tostring(comments,
method='text', encoding=unicode)
mi.comments = re.sub('<([pP]|DIV)>', '\n\n', mi.comments)
mi.comments = re.sub('</?[iI]>', '*', mi.comments)
mi.comments = re.sub('</?[bB]>', '**', mi.comments)
mi.comments = re.sub('<BR>', '\n\n', mi.comments)
mi.comments = re.sub('<[^>]+>', '', mi.comments)
mi.comments = mi.comments.strip()
mi.comments = _('EDITORIAL REVIEW')+':\n\n'+mi.comments
mi = Metadata(title, authors)
if not isbn:
return mi
isbn = check_isbn(isbn)
if not isbn:
return mi
br = browser()
asin = to_asin(br, isbn)
if asin:
if get_metadata(br, asin, mi):
return mi
from calibre.ebooks.metadata.xisbn import xisbn
for i in xisbn.get_associated_isbns(isbn):
asin = to_asin(br, i)
if get_metadata(br, asin, mi):
return mi
return mi
def get_metadata(br, asin, mi):
q = 'http://amzn.com/'+asin
try:
raw = br.open_novisit(q).read()
except Exception, e:
if callable(getattr(e, 'getcode', None)) and \
e.getcode() == 404:
return False
raise
if '<title>404 - ' in raw:
return False
raw = xml_to_unicode(raw, strip_encoding_pats=True,
resolve_entities=True)[0]
root = html.fromstring(raw)
ratings = root.xpath('//form[@id="handleBuy"]/descendant::*[@class="asinReviewsSummary"]')
if ratings:
pat = re.compile(r'([0-9.]+) out of (\d+) stars')
r = ratings[0]
for elem in r.xpath('descendant::*[@title]'):
t = elem.get('title')
m = pat.match(t)
if m is not None:
try:
mi.rating = float(m.group(1))/float(m.group(2)) * 5
break
except:
pass
desc = root.xpath('//div[@id="productDescription"]/*[@class="content"]')
if desc:
desc = desc[0]
for c in desc.xpath('descendant::*[@class="seeAll" or'
' @class="emptyClear" or @href]'):
c.getparent().remove(c)
desc = html.tostring(desc, method='html', encoding=unicode).strip()
# remove all attributes from tags
desc = re.sub(r'<([a-zA-Z0-9]+)\s[^>]+>', r'<\1>', desc)
# Collapse whitespace
desc = re.sub('\n+', '\n', desc)
desc = re.sub(' +', ' ', desc)
# Remove the notice about text referring to out of print editions
desc = re.sub(r'(?s)<em>--This text ref.*?</em>', '', desc)
# Remove comments
desc = re.sub(r'(?s)<!--.*?-->', '', desc)
mi.comments = desc
return True
def main(args=sys.argv):
print get_social_metadata(None, None, None, '9781416551720')
# Test xisbn
print get_social_metadata('Learning Python', None, None, '8324616489')
print
# Test sophisticated comment formatting
print get_social_metadata('Swan Thieves', None, None, '9780316065795')
print
# Random tests
print get_social_metadata('Star Trek: Destiny: Mere Mortals', None, None, '9781416551720')
print
print get_social_metadata('The Great Gatsby', None, None, '0743273567')
return 0
if __name__ == '__main__':

View File

@ -22,7 +22,8 @@ def get_metadata(stream):
'xlink':XLINK_NS})
tostring = lambda x : etree.tostring(x, method='text',
encoding=unicode).strip()
root = etree.fromstring(stream.read())
parser = etree.XMLParser(recover=True, no_network=True)
root = etree.fromstring(stream.read(), parser=parser)
authors, author_sort = [], None
for au in XPath('//fb2:author')(root):
fname = lname = author = None

View File

@ -12,7 +12,7 @@ import os, time, sys, shutil
from calibre.utils.ipc.job import ParallelJob
from calibre.utils.ipc.server import Server
from calibre.ptempfile import PersistentTemporaryDirectory
from calibre.ptempfile import PersistentTemporaryDirectory, TemporaryDirectory
from calibre import prints
from calibre.constants import filesystem_encoding
@ -21,51 +21,76 @@ def debug(*args):
prints(*args)
sys.stdout.flush()
def read_metadata_(task, tdir, notification=lambda x,y:x):
def serialize_metadata_for(formats, tdir, id_):
from calibre.ebooks.metadata.meta import metadata_from_formats
from calibre.ebooks.metadata.opf2 import metadata_to_opf
mi = metadata_from_formats(formats)
mi.cover = None
cdata = None
if mi.cover_data:
cdata = mi.cover_data[-1]
mi.cover_data = None
if not mi.application_id:
mi.application_id = '__calibre_dummy__'
with open(os.path.join(tdir, '%s.opf'%id_), 'wb') as f:
f.write(metadata_to_opf(mi))
if cdata:
with open(os.path.join(tdir, str(id_)), 'wb') as f:
f.write(cdata)
def read_metadata_(task, tdir, notification=lambda x,y:x):
with TemporaryDirectory() as mdir:
do_read_metadata(task, tdir, mdir, notification)
def do_read_metadata(task, tdir, mdir, notification):
from calibre.customize.ui import run_plugins_on_import
for x in task:
try:
id, formats = x
id_, formats = x
except:
continue
try:
if isinstance(formats, basestring): formats = [formats]
mi = metadata_from_formats(formats)
mi.cover = None
cdata = None
if mi.cover_data:
cdata = mi.cover_data[-1]
mi.cover_data = None
if not mi.application_id:
mi.application_id = '__calibre_dummy__'
with open(os.path.join(tdir, '%s.opf'%id), 'wb') as f:
f.write(metadata_to_opf(mi))
if cdata:
with open(os.path.join(tdir, str(id)), 'wb') as f:
f.write(cdata)
import_map = {}
fmts, metadata_fmts = [], []
for format in formats:
mfmt = format
name, ext = os.path.splitext(os.path.basename(format))
nfp = run_plugins_on_import(format)
if nfp is None:
nfp = format
nfp = os.path.abspath(nfp)
if not nfp or nfp == format or not os.access(nfp, os.R_OK):
nfp = None
else:
# Ensure that the filename is preserved so that
# reading metadata from filename is not broken
nfp = os.path.abspath(nfp)
nfext = os.path.splitext(nfp)[1]
mfmt = os.path.join(mdir, name + nfext)
shutil.copyfile(nfp, mfmt)
metadata_fmts.append(mfmt)
fmts.append(nfp)
serialize_metadata_for(metadata_fmts, tdir, id_)
for format, nfp in zip(formats, fmts):
if not nfp:
continue
if isinstance(nfp, unicode):
nfp.encode(filesystem_encoding)
x = lambda j : os.path.abspath(os.path.normpath(os.path.normcase(j)))
if x(nfp) != x(format) and os.access(nfp, os.R_OK|os.W_OK):
fmt = os.path.splitext(format)[1].replace('.', '').lower()
nfmt = os.path.splitext(nfp)[1].replace('.', '').lower()
dest = os.path.join(tdir, '%s.%s'%(id, nfmt))
dest = os.path.join(tdir, '%s.%s'%(id_, nfmt))
shutil.copyfile(nfp, dest)
import_map[fmt] = dest
os.remove(nfp)
if import_map:
with open(os.path.join(tdir, str(id)+'.import'), 'wb') as f:
with open(os.path.join(tdir, str(id_)+'.import'), 'wb') as f:
for fmt, nfp in import_map.items():
f.write(fmt+':'+nfp+'\n')
notification(0.5, id)
notification(0.5, id_)
except:
import traceback
with open(os.path.join(tdir, '%s.error'%id), 'wb') as f:
with open(os.path.join(tdir, '%s.error'%id_), 'wb') as f:
f.write(traceback.format_exc())
class Progress(object):

View File

@ -0,0 +1,80 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
__license__ = 'GPL v3'
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import threading, re, json
from calibre import browser
class xISBN(object):
QUERY = 'http://xisbn.worldcat.org/webservices/xid/isbn/%s?method=getEditions&format=json&fl=form,year,lang,ed'
def __init__(self):
self.lock = threading.RLock()
self._data = []
self._map = {}
self.br = browser()
self.isbn_pat = re.compile(r'[^0-9X]', re.IGNORECASE)
def purify(self, isbn):
return self.isbn_pat.sub('', isbn.upper())
def fetch_data(self, isbn):
url = self.QUERY%isbn
data = self.br.open_novisit(url).read()
data = json.loads(data)
if data.get('stat', None) != 'ok':
return []
data = data.get('list', [])
ans = []
for rec in data:
forms = rec.get('form', [])
# Only get books, not audio/video
forms = [x for x in forms if x in ('BA', 'BC', 'BB', 'DA')]
if forms:
ans.append(rec)
return ans
def get_data(self, isbn):
isbn = self.purify(isbn)
with self.lock:
if isbn not in self._map:
try:
data = self.fetch_data(isbn)
except:
import traceback
traceback.print_exc()
data = []
id_ = len(self._data)
self._data.append(data)
for rec in data:
for i in rec.get('isbn', []):
self._map[i] = id_
self._map[isbn] = id_
return self._data[self._map[isbn]]
def get_associated_isbns(self, isbn):
data = self.get_data(isbn)
ans = set([])
for rec in data:
for i in rec.get('isbn', []):
ans.add(i)
return ans
xisbn = xISBN()
if __name__ == '__main__':
import sys
isbn = sys.argv[-1]
print xisbn.get_data(isbn)
print
print xisbn.get_associated_isbns(isbn)

View File

@ -27,6 +27,8 @@ TABLE_TAGS = set(['table', 'tr', 'td', 'th', 'caption'])
SPECIAL_TAGS = set(['hr', 'br'])
CONTENT_TAGS = set(['img', 'hr', 'br'])
NOT_VTAGS = HEADER_TAGS | NESTABLE_TAGS | TABLE_TAGS | SPECIAL_TAGS | \
CONTENT_TAGS
PAGE_BREAKS = set(['always', 'left', 'right'])
COLLAPSE = re.compile(r'[ \t\r\n\v]+')
@ -57,8 +59,6 @@ class FormatState(object):
self.indent = 0.
self.fsize = 3
self.ids = set()
self.valign = 'baseline'
self.nest = False
self.italic = False
self.bold = False
self.strikethrough = False
@ -76,7 +76,6 @@ class FormatState(object):
and self.italic == other.italic \
and self.bold == other.bold \
and self.href == other.href \
and self.valign == other.valign \
and self.preserve == other.preserve \
and self.family == other.family \
and self.bgcolor == other.bgcolor \
@ -224,7 +223,6 @@ class MobiMLizer(object):
return
if not pstate or istate != pstate:
inline = para
valign = istate.valign
fsize = istate.fsize
href = istate.href
if not href:
@ -234,19 +232,8 @@ class MobiMLizer(object):
else:
inline = etree.SubElement(inline, XHTML('a'), href=href)
bstate.anchor = inline
if valign == 'super':
parent = inline
if istate.nest and bstate.inline is not None:
parent = bstate.inline
istate.nest = False
inline = etree.SubElement(parent, XHTML('sup'))
elif valign == 'sub':
parent = inline
if istate.nest and bstate.inline is not None:
parent = bstate.inline
istate.nest = False
inline = etree.SubElement(parent, XHTML('sub'))
elif fsize != 3:
if fsize != 3:
inline = etree.SubElement(inline, XHTML('font'),
size=str(fsize))
if istate.family == 'monospace':
@ -279,7 +266,8 @@ class MobiMLizer(object):
else:
inline.append(item)
def mobimlize_elem(self, elem, stylizer, bstate, istates):
def mobimlize_elem(self, elem, stylizer, bstate, istates,
ignore_valign=False):
if not isinstance(elem.tag, basestring) \
or namespace(elem.tag) != XHTML_NS:
return
@ -351,15 +339,6 @@ class MobiMLizer(object):
istate.family = 'sans-serif'
else:
istate.family = 'serif'
valign = style['vertical-align']
if valign in ('super', 'text-top') or asfloat(valign) > 0:
istate.nest = istate.valign in ('sub', 'super')
istate.valign = 'super'
elif valign == 'sub' or asfloat(valign) < 0:
istate.nest = istate.valign in ('sub', 'super')
istate.valign = 'sub'
else:
istate.valign = 'baseline'
if 'id' in elem.attrib:
istate.ids.add(elem.attrib['id'])
if 'name' in elem.attrib:
@ -407,6 +386,30 @@ class MobiMLizer(object):
text = None
else:
text = COLLAPSE.sub(' ', elem.text)
valign = style['vertical-align']
not_baseline = valign in ('super', 'sub', 'text-top',
'text-bottom')
vtag = 'sup' if valign in ('super', 'text-top') else 'sub'
if not_baseline and not ignore_valign and tag not in NOT_VTAGS and not isblock:
nroot = etree.Element(XHTML('html'), nsmap=MOBI_NSMAP)
vbstate = BlockState(etree.SubElement(nroot, XHTML('body')))
vbstate.para = etree.SubElement(vbstate.body, XHTML('p'))
self.mobimlize_elem(elem, stylizer, vbstate, istates,
ignore_valign=True)
if len(istates) > 0:
istates.pop()
if len(istates) == 0:
istates.append(FormatState())
at_start = bstate.para is None
if at_start:
self.mobimlize_content('span', '', bstate, istates)
parent = bstate.para if bstate.inline is None else bstate.inline
if parent is not None:
vtag = etree.SubElement(parent, XHTML(vtag))
for child in vbstate.para:
vtag.append(child)
return
if text or tag in CONTENT_TAGS or tag in NESTABLE_TAGS:
self.mobimlize_content(tag, text, bstate, istates)
for child in elem:
@ -421,6 +424,8 @@ class MobiMLizer(object):
tail = COLLAPSE.sub(' ', child.tail)
if tail:
self.mobimlize_content(tag, tail, bstate, istates)
if bstate.content and style['page-break-after'] in PAGE_BREAKS:
bstate.pbreak = True
if isblock:

View File

@ -2043,12 +2043,16 @@ class MobiWriter(object):
else :
self._oeb.logger.info("chapterCount: %d" % self._chapterCount)
if True:
rec_count = len(self._ctoc_records)
self._oeb.logger.info(" CNCX utilization: %d %s %.0f%% full" % \
(rec_count + 1, 'records, last record' if rec_count else 'record,', len(self._ctoc.getvalue())/655) )
# Apparently the CTOC must end with a null byte
self._ctoc.write('\0')
return align_block(self._ctoc.getvalue())
ctoc = self._ctoc.getvalue()
rec_count = len(self._ctoc_records)
self._oeb.logger.info(" CNCX utilization: %d %s %.0f%% full" % \
(rec_count + 1, 'records, last record' if rec_count else 'record,',
len(ctoc)/655) )
return align_block(ctoc)
def _write_periodical_node(self, indxt, indices, index, offset, length, count, firstSection, lastSection) :
pos = 0xc0 + indxt.tell()

View File

@ -25,6 +25,7 @@ from calibre.translations.dynamic import translate
from calibre.ebooks.chardet import xml_to_unicode
from calibre.ebooks.oeb.entitydefs import ENTITYDEFS
from calibre.ebooks.conversion.preprocess import CSSPreProcessor
from calibre import isbytestring
RECOVER_PARSER = etree.XMLParser(recover=True, no_network=True)
@ -404,7 +405,8 @@ class DirContainer(object):
def __init__(self, path, log):
self.log = log
path = unicode(path)
if isbytestring(path):
path = path.decode(filesystem_encoding)
ext = os.path.splitext(path)[1].lower()
if ext == '.opf':
self.opfname = os.path.basename(path)

View File

@ -20,20 +20,10 @@ class SNBOutput(OutputFormatPlugin):
file_type = 'snb'
options = set([
# OptionRecommendation(name='newline', recommended_value='system',
# level=OptionRecommendation.LOW,
# short_switch='n', choices=TxtNewlines.NEWLINE_TYPES.keys(),
# help=_('Type of newline to use. Options are %s. Default is \'system\'. '
# 'Use \'old_mac\' for compatibility with Mac OS 9 and earlier. '
# 'For Mac OS X use \'unix\'. \'system\' will default to the newline '
# 'type used by this OS.') % sorted(TxtNewlines.NEWLINE_TYPES.keys())),
OptionRecommendation(name='snb_output_encoding', recommended_value='utf-8',
level=OptionRecommendation.LOW,
help=_('Specify the character encoding of the output document. ' \
'The default is utf-8.')),
# OptionRecommendation(name='inline_toc',
# recommended_value=False, level=OptionRecommendation.LOW,
# help=_('Add Table of Contents to beginning of the book.')),
OptionRecommendation(name='snb_max_line_length',
recommended_value=0, level=OptionRecommendation.LOW,
help=_('The maximum number of characters per line. This splits on '
@ -41,10 +31,18 @@ class SNBOutput(OutputFormatPlugin):
'the line will be broken at the space after and will exceed the '
'specified value. Also, there is a minimum of 25 characters. '
'Use 0 to disable line splitting.')),
# OptionRecommendation(name='force_max_line_length',
# recommended_value=False, level=OptionRecommendation.LOW,
# help=_('Force splitting on the max-line-length value when no space '
# 'is present. Also allows max-line-length to be below the minimum')),
OptionRecommendation(name='snb_insert_empty_line',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Specify whether or not to insert an empty line between '
'two paragraphs.')),
OptionRecommendation(name='snb_indent_first_line',
recommended_value=True, level=OptionRecommendation.LOW,
help=_('Specify whether or not to insert two space characters '
'to indent the first line of each paragraph.')),
OptionRecommendation(name='snb_hide_chapter_name',
recommended_value=False, level=OptionRecommendation.LOW,
help=_('Specify whether or not to hide the chapter title for each '
'chapter. Useful for image-only output (eg. comics).')),
])
def convert(self, oeb_book, output_path, input_plugin, opts, log):
@ -230,7 +228,7 @@ class SNBOutput(OutputFormatPlugin):
img.load(imageData)
(x,y) = img.size
if self.opts:
SCREEN_Y, SCREEN_X = self.opts.output_profile.comic_screen_size
SCREEN_X, SCREEN_Y = self.opts.output_profile.comic_screen_size
else:
SCREEN_X = 540
SCREEN_Y = 700

View File

@ -88,7 +88,10 @@ class SNBMLizer(object):
trees = { }
for subitem, subtitle in self.subitems:
snbcTree = etree.Element("snbc")
etree.SubElement(etree.SubElement(snbcTree, "head"), "title").text = subtitle
snbcHead = etree.SubElement(snbcTree, "head")
etree.SubElement(snbcHead, "title").text = subtitle
if self.opts and self.opts.snb_hide_chapter_name:
etree.SubElement(snbcHead, "hidetitle").text = u"true"
etree.SubElement(snbcTree, "body")
trees[subitem] = snbcTree
output.append(u'%s%s\n\n' % (CALIBRE_SNB_BM_TAG, ""))
@ -96,27 +99,37 @@ class SNBMLizer(object):
output = self.cleanup_text(u''.join(output))
subitem = ''
bodyTree = trees[subitem].find(".//body")
for line in output.splitlines():
if not line.find(CALIBRE_SNB_PRE_TAG) == 0:
line = line.strip(u' \t\n\r\u3000')
else:
etree.SubElement(trees[subitem].find(".//body"), "text").text = \
etree.SubElement(bodyTree, "text").text = \
etree.CDATA(line[len(CALIBRE_SNB_PRE_TAG):])
continue
if len(line) != 0:
if line.find(CALIBRE_SNB_IMG_TAG) == 0:
prefix = ProcessFileName(os.path.dirname(self.item.href))
if prefix != '':
etree.SubElement(trees[subitem].find(".//body"), "img").text = \
etree.SubElement(bodyTree, "img").text = \
prefix + '_' + line[len(CALIBRE_SNB_IMG_TAG):]
else:
etree.SubElement(trees[subitem].find(".//body"), "img").text = \
etree.SubElement(bodyTree, "img").text = \
line[len(CALIBRE_SNB_IMG_TAG):]
elif line.find(CALIBRE_SNB_BM_TAG) == 0:
subitem = line[len(CALIBRE_SNB_BM_TAG):]
bodyTree = trees[subitem].find(".//body")
else:
etree.SubElement(trees[subitem].find(".//body"), "text").text = \
etree.CDATA(unicode(u'\u3000\u3000' + line))
if self.opts and self.opts.snb_indent_first_line:
prefix = u'\u3000\u3000'
else:
prefix = u''
etree.SubElement(bodyTree, "text").text = \
etree.CDATA(unicode(prefix + line))
if self.opts and self.opts.snb_insert_empty_line:
etree.SubElement(bodyTree, "text").text = \
etree.CDATA(u'')
return trees
def remove_newlines(self, text):

View File

@ -255,7 +255,7 @@ def error_dialog(parent, title, msg, det_msg='', show=False,
return d
def question_dialog(parent, title, msg, det_msg='', show_copy_button=True,
buttons=QMessageBox.Yes|QMessageBox.No):
buttons=QMessageBox.Yes|QMessageBox.No, yes_button=QMessageBox.Yes):
d = MessageBox(QMessageBox.Question, title, msg, buttons,
parent, det_msg)
d.setIconPixmap(QPixmap(I('dialog_question.png')))
@ -263,7 +263,7 @@ def question_dialog(parent, title, msg, det_msg='', show_copy_button=True,
if not show_copy_button:
d.cb.setVisible(False)
return d.exec_() == QMessageBox.Yes
return d.exec_() == yes_button
def info_dialog(parent, title, msg, det_msg='', show=False):
d = MessageBox(QMessageBox.Information, title, msg, QMessageBox.Ok,
@ -399,6 +399,7 @@ class FileIconProvider(QFileIconProvider):
'fb2' : 'fb2',
'rtf' : 'rtf',
'odt' : 'odt',
'snb' : 'snb',
}
def __init__(self):

View File

@ -111,7 +111,7 @@ class ViewAction(InterfaceAction):
'books at once can be slow and have a negative effect on the '
'responsiveness of your computer. Once started the process '
'cannot be stopped until complete. Do you wish to continue?'
) % num)
) % num, show_copy_button=False)
def view_folder(self, *args):
rows = self.gui.current_view().selectionModel().selectedRows()

View File

@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
__license__ = 'GPL 3'
__copyright__ = '2009, John Schember <john@nachtimwald.com>'
__copyright__ = '2010, Li Fanxi <lifanxi@freemindworld.com>'
__docformat__ = 'restructuredtext en'
from calibre.gui2.convert.snb_output_ui import Ui_Form
@ -18,18 +18,9 @@ class PluginWidget(Widget, Ui_Form):
def __init__(self, parent, get_option, get_help, db=None, book_id=None):
Widget.__init__(self, parent,
[])
['snb_insert_empty_line', 'snb_indent_first_line',
'snb_hide_chapter_name',])
self.db, self.book_id = db, book_id
self.initialize_options(get_option, get_help, db, book_id)
# default = self.opt_newline.currentText()
# global newline_model
# if newline_model is None:
# newline_model = BasicComboModel(TxtNewlines.NEWLINE_TYPES.keys())
# self.newline_model = newline_model
# self.opt_newline.setModel(self.newline_model)
# default_index = self.opt_newline.findText(default)
# system_index = self.opt_newline.findText('system')
# self.opt_newline.setCurrentIndex(default_index if default_index != -1 else system_index if system_index != -1 else 0)

View File

@ -13,60 +13,41 @@
<property name="windowTitle">
<string>Form</string>
</property>
<layout class="QGridLayout" name="gridLayout">
<!-- <item row="0" column="0"> -->
<!-- <widget class="QLabel" name="label"> -->
<!-- <property name="text"> -->
<!-- <string>&amp;Line ending style:</string> -->
<!-- </property> -->
<!-- <property name="buddy"> -->
<!-- <cstring>opt_newline</cstring> -->
<!-- </property> -->
<!-- </widget> -->
<!-- </item> -->
<!-- <item row="0" column="1"> -->
<!-- <widget class="QComboBox" name="opt_newline"/> -->
<!-- </item> -->
<!-- <item row="4" column="0"> -->
<!-- <spacer name="verticalSpacer"> -->
<!-- <property name="orientation"> -->
<!-- <enum>Qt::Vertical</enum> -->
<!-- </property> -->
<!-- <property name="sizeHint" stdset="0"> -->
<!-- <size> -->
<!-- <width>20</width> -->
<!-- <height>246</height> -->
<!-- </size> -->
<!-- </property> -->
<!-- </spacer> -->
<!-- </item> -->
<!-- <item row="3" column="0" colspan="2"> -->
<!-- <widget class="QCheckBox" name="opt_inline_toc"> -->
<!-- <property name="text"> -->
<!-- <string>&amp;Inline TOC</string> -->
<!-- </property> -->
<!-- </widget> -->
<!-- </item> -->
<!-- <item row="1" column="1"> -->
<!-- <widget class="QSpinBox" name="opt_max_line_length"/> -->
<!-- </item> -->
<!-- <item row="1" column="0"> -->
<!-- <widget class="QLabel" name="label_2"> -->
<!-- <property name="text"> -->
<!-- <string>&amp;Maximum line length:</string> -->
<!-- </property> -->
<!-- <property name="buddy"> -->
<!-- <cstring>opt_max_line_length</cstring> -->
<!-- </property> -->
<!-- </widget> -->
<!-- </item> -->
<!-- <item row="2" column="0" colspan="2"> -->
<!-- <widget class="QCheckBox" name="opt_force_max_line_length"> -->
<!-- <property name="text"> -->
<!-- <string>Force maximum line length</string> -->
<!-- </property> -->
<!-- </widget> -->
<!-- </item> -->
<layout class="QGridLayout" name="gridLayout" rowstretch="0,0,0,0,0">
<item row="4" column="0">
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>40</height>
</size>
</property>
</spacer>
</item>
<item row="3" column="0">
<widget class="QCheckBox" name="opt_snb_hide_chapter_name">
<property name="text">
<string>Hide chapter name</string>
</property>
</widget>
</item>
<item row="2" column="0">
<widget class="QCheckBox" name="opt_snb_indent_first_line">
<property name="text">
<string>Insert space before the first line for each paragraph</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QCheckBox" name="opt_snb_insert_empty_line">
<property name="text">
<string>Insert empty line between paragraphs</string>
</property>
</widget>
</item>
</layout>
</widget>
<resources/>

View File

@ -489,7 +489,7 @@ class DeviceMenu(QMenu): # {{{
for actions, desc in (
(basic_actions, ''),
(delete_actions, _('Send and delete from library')),
(specific_actions, _('Send specific format'))
(specific_actions, _('Send specific format to'))
):
mdest = menu
if actions is not basic_actions:
@ -1029,7 +1029,7 @@ class DeviceMixin(object): # {{{
to_s = [account]
subjects = [_('News:')+' '+mi.title]
texts = [_('Attached is the')+' '+mi.title]
attachment_names = [mi.title+os.path.splitext(attachment)[1]]
attachment_names = [ascii_filename(mi.title)+os.path.splitext(attachment)[1]]
attachments = [attachment]
jobnames = ['%s:%s'%(id, mi.title)]
remove = [id] if config['delete_news_from_library_on_upload']\

View File

@ -55,12 +55,16 @@ class CheckLibraryDialog(QDialog):
h.addWidget(ln)
self.name_ignores = QLineEdit()
self.name_ignores.setText(db.prefs.get('check_library_ignore_names', ''))
self.name_ignores.setToolTip(
_('Enter comma-separated standard file name wildcards, such as synctoy*.dat'))
ln.setBuddy(self.name_ignores)
h.addWidget(self.name_ignores)
le = QLabel(_('Extensions to ignore'))
h.addWidget(le)
self.ext_ignores = QLineEdit()
self.ext_ignores.setText(db.prefs.get('check_library_ignore_extensions', ''))
self.ext_ignores.setToolTip(
_('Enter comma-separated extensions without a leading dot. Used only in book folders'))
le.setBuddy(self.ext_ignores)
h.addWidget(self.ext_ignores)
self._layout.addLayout(h)

View File

@ -571,6 +571,10 @@ class MetadataBulkDialog(QDialog, Ui_MetadataBulkDialog):
self.initalize_authors()
self.initialize_series()
self.initialize_publisher()
for x in ('authors', 'publisher', 'series'):
x = getattr(self, x)
x.setSizeAdjustPolicy(x.AdjustToMinimumContentsLengthWithIcon)
x.setMinimumContentsLength(25)
def initalize_authors(self):
all_authors = self.db.all_authors()

View File

@ -678,6 +678,19 @@ nothing should be put between the original text and the inserted text</string>
<item row="8" column="2">
<widget class="QLineEdit" name="test_result"/>
</item>
<item row="25" column="0" colspan="2">
<spacer name="verticalSpacer_2">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>5</height>
</size>
</property>
</spacer>
</item>
</layout>
</widget>
</widget>

View File

@ -9,11 +9,11 @@ add/remove formats
import os, re, time, traceback, textwrap
from PyQt4.Qt import SIGNAL, QObject, Qt, QTimer, QThread, QDate, \
QPixmap, QListWidgetItem, QDialog, pyqtSignal
QPixmap, QListWidgetItem, QDialog, pyqtSignal, QMessageBox
from calibre.gui2 import error_dialog, file_icon_provider, dynamic, \
choose_files, choose_images, ResizableDialog, \
warning_dialog
warning_dialog, question_dialog
from calibre.gui2.dialogs.metadata_single_ui import Ui_MetadataSingleDialog
from calibre.gui2.dialogs.fetch_metadata import FetchMetadata
from calibre.gui2.dialogs.tag_editor import TagEditor
@ -608,9 +608,16 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
def edit_tags(self):
if self.tags.text() != self.original_tags:
error_dialog(self, _('Cannot use tag editor'),
_('The tags editor cannot be used if you have modified the tags')).exec_()
return
if question_dialog(self, _('Tags changed'),
_('You have changed the tags. In order to use the tags'
' editor, you must either discard or apply these '
'changes'), show_copy_button=False,
buttons=QMessageBox.Apply|QMessageBox.Discard,
yes_button=QMessageBox.Apply):
self.apply_tags(commit=True, notify=True)
self.original_tags = unicode(self.tags.text())
else:
self.tags.setText(self.original_tags)
d = TagEditor(self, self.db, self.row)
d.exec_()
if d.result() == QDialog.Accepted:
@ -764,6 +771,10 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
self.series.setCurrentIndex(i)
break
def apply_tags(self, commit=False, notify=False):
self.db.set_tags(self.id, [x.strip() for x in
unicode(self.tags.text()).split(',')],
notify=notify, commit=commit)
def accept(self):
cf = getattr(self, 'cover_fetcher', None)
@ -787,11 +798,10 @@ class MetadataSingleDialog(ResizableDialog, Ui_MetadataSingleDialog):
notify=False, commit=False)
self.db.set_rating(self.id, 2*self.rating.value(), notify=False,
commit=False)
self.apply_tags()
self.db.set_publisher(self.id,
unicode(self.publisher.currentText()).strip(),
notify=False, commit=False)
self.db.set_tags(self.id, [x.strip() for x in
unicode(self.tags.text()).split(',')], notify=False, commit=False)
self.db.set_series(self.id,
unicode(self.series.currentText()).strip(), notify=False,
commit=False)

View File

@ -16,7 +16,7 @@ from PyQt4.Qt import QIcon, QFont, QLabel, QListWidget, QAction, \
QTimer, QRect
from calibre.gui2 import NONE, error_dialog, pixmap_to_data, gprefs
from calibre.constants import isosx
from calibre.gui2.filename_pattern_ui import Ui_Form
from calibre import fit_image
from calibre.utils.fonts import fontconfig
@ -303,7 +303,8 @@ class FontFamilyModel(QAbstractListModel):
return NONE
if role == Qt.DisplayRole:
return QVariant(family)
if role == Qt.FontRole:
if not isosx and role == Qt.FontRole:
# Causes a Qt crash with some fonts on OS X
return QVariant(QFont(family))
return NONE

View File

@ -219,6 +219,13 @@ class EZReaderPP(HanlinV5):
manufacturer = 'Astak'
id = 'ezreader_pp'
class Bambook(Device):
name = 'Sanda Bambook'
output_format = 'SNB'
manufacturer = 'Sanda'
id = 'bambook'
output_profile = 'bambook'
# }}}
def get_devices():

View File

@ -2,7 +2,7 @@ __license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal <kovid at kovidgoyal.net>'
''' Code to manage ebook library'''
def db():
def db(path=None):
from calibre.library.database2 import LibraryDatabase2
from calibre.utils.config import prefs
return LibraryDatabase2(prefs['library_path'])
return LibraryDatabase2(path if path else prefs['library_path'])

View File

@ -5,7 +5,7 @@ __license__ = 'GPL v3'
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import re, os, traceback
import re, os, traceback, fnmatch
from calibre import isbytestring
from calibre.constants import filesystem_encoding
@ -66,13 +66,19 @@ class CheckLibrary(object):
return self.failed_folders or self.mismatched_dirs or \
self.conflicting_custom_cols or self.failed_restores
def ignore_name(self, filename):
for filespec in self.ignore_names:
if fnmatch.fnmatch(filename, filespec):
return True
return False;
def scan_library(self, name_ignores, extension_ignores):
self.ignore_names = frozenset(name_ignores)
self.ignore_ext = frozenset(['.'+ e for e in extension_ignores])
lib = self.src_library_path
for auth_dir in os.listdir(lib):
if auth_dir in self.ignore_names or auth_dir == 'metadata.db':
if self.ignore_name(auth_dir) or auth_dir == 'metadata.db':
continue
auth_path = os.path.join(lib, auth_dir)
# First check: author must be a directory
@ -85,7 +91,7 @@ class CheckLibrary(object):
# Look for titles in the author directories
found_titles = False
for title_dir in os.listdir(auth_path):
if title_dir in self.ignore_names:
if self.ignore_name(title_dir):
continue
title_path = os.path.join(auth_path, title_dir)
db_path = os.path.join(auth_dir, title_dir)

View File

@ -44,6 +44,10 @@ def server_config(defaults=None):
'by first letter when there are more than this number '
'of items. Default: %default. Set to a large number '
'to disable grouping.'))
c.add_opt('url_prefix', ['--url-prefix'], default='',
help=_('Prefix to prepend to all URLs. Useful for reverse'
'proxying to this server from Apache/nginx/etc.'))
return c
def custom_fields_to_display(db):

View File

@ -28,16 +28,21 @@ from calibre.library.server.browse import BrowseServer
class DispatchController(object): # {{{
def __init__(self):
def __init__(self, prefix, wsgi=False):
self.dispatcher = cherrypy.dispatch.RoutesDispatcher()
self.funcs = []
self.seen = set([])
self.prefix = prefix if prefix else ''
if wsgi:
self.prefix = ''
def __call__(self, name, route, func, **kwargs):
if name in self.seen:
raise NameError('Route name: '+ repr(name) + ' already used')
self.seen.add(name)
kwargs['action'] = 'f_%d'%len(self.funcs)
if route != '/':
route = self.prefix + route
self.dispatcher.connect(name, route, self, **kwargs)
self.funcs.append(expose(func))
@ -55,16 +60,17 @@ class DispatchController(object): # {{{
# }}}
class BonJour(SimplePlugin):
class BonJour(SimplePlugin): # {{{
def __init__(self, engine, port=8080):
def __init__(self, engine, port=8080, prefix=''):
SimplePlugin.__init__(self, engine)
self.port = port
self.prefix = prefix
def start(self):
try:
publish_zeroconf('Books in calibre', '_stanza._tcp',
self.port, {'path':'/stanza'})
self.port, {'path':self.prefix+'/stanza'})
except:
import traceback
cherrypy.log.error('Failed to start BonJour:')
@ -85,13 +91,16 @@ class BonJour(SimplePlugin):
cherrypy.engine.bonjour = BonJour(cherrypy.engine)
# }}}
class LibraryServer(ContentServer, MobileServer, XMLServer, OPDSServer, Cache,
BrowseServer):
server_name = __appname__ + '/' + __version__
def __init__(self, db, opts, embedded=False, show_tracebacks=True):
def __init__(self, db, opts, embedded=False, show_tracebacks=True,
wsgi=False):
self.is_wsgi = bool(wsgi)
self.opts = opts
self.embedded = embedded
self.state_callback = None
@ -102,6 +111,7 @@ class LibraryServer(ContentServer, MobileServer, XMLServer, OPDSServer, Cache,
self.default_cover = open(P('content_server/default_cover.jpg'), 'rb').read()
cherrypy.engine.bonjour.port = opts.port
cherrypy.engine.bonjour.prefix = opts.url_prefix
Cache.__init__(self)
@ -118,25 +128,36 @@ class LibraryServer(ContentServer, MobileServer, XMLServer, OPDSServer, Cache,
'server.socket_timeout' : opts.timeout, #seconds
'server.thread_pool' : opts.thread_pool, # number of threads
})
if embedded:
if embedded or wsgi:
cherrypy.config.update({'engine.SIGHUP' : None,
'engine.SIGTERM' : None,})
self.config = {'global': {
'tools.gzip.on' : True,
'tools.gzip.mime_types': ['text/html', 'text/plain', 'text/xml', 'text/javascript', 'text/css'],
}}
if opts.password:
self.config['/'] = {
'tools.digest_auth.on' : True,
'tools.digest_auth.realm' : (_('Password to access your calibre library. Username is ') + opts.username.strip()).encode('ascii', 'replace'),
'tools.digest_auth.users' : {opts.username.strip():opts.password.strip()},
}
self.config = {}
self.is_running = False
self.exception = None
self.setup_loggers()
cherrypy.engine.bonjour.subscribe()
if not wsgi:
self.setup_loggers()
cherrypy.engine.bonjour.subscribe()
self.config['global'] = {
'tools.gzip.on' : True,
'tools.gzip.mime_types': ['text/html', 'text/plain',
'text/xml', 'text/javascript', 'text/css'],
}
if opts.password:
self.config['/'] = {
'tools.digest_auth.on' : True,
'tools.digest_auth.realm' : (
_('Password to access your calibre library. Username is ')
+ opts.username.strip()),
'tools.digest_auth.users' : {opts.username.strip():opts.password.strip()},
}
self.__dispatcher__ = DispatchController(self.opts.url_prefix, wsgi)
for x in self.__class__.__bases__:
if hasattr(x, 'add_routes'):
x.add_routes(self, self.__dispatcher__)
root_conf = self.config.get('/', {})
root_conf['request.dispatch'] = self.__dispatcher__.dispatcher
self.config['/'] = root_conf
def set_database(self, db):
self.db = db
@ -177,14 +198,6 @@ class LibraryServer(ContentServer, MobileServer, XMLServer, OPDSServer, Cache,
def start(self):
self.is_running = False
d = DispatchController()
for x in self.__class__.__bases__:
if hasattr(x, 'add_routes'):
x.add_routes(self, d)
root_conf = self.config.get('/', {})
root_conf['request.dispatch'] = d.dispatcher
self.config['/'] = root_conf
cherrypy.tree.mount(root=None, config=self.config)
try:
try:

View File

@ -22,7 +22,7 @@ from calibre.library.comments import comments_to_html
from calibre.library.server import custom_fields_to_display
from calibre.library.field_metadata import category_icon_map
def render_book_list(ids, suffix=''): # {{{
def render_book_list(ids, prefix, suffix=''): # {{{
pages = []
num = len(ids)
pos = 0
@ -35,11 +35,11 @@ def render_book_list(ids, suffix=''): # {{{
page_template = u'''\
<div class="page" id="page{0}">
<div class="load_data" title="{1}">
<span class="url" title="/browse/booklist_page"></span>
<span class="url" title="{prefix}/browse/booklist_page"></span>
<span class="start" title="{start}"></span>
<span class="end" title="{end}"></span>
</div>
<div class="loading"><img src="/static/loading.gif" /> {2}</div>
<div class="loading"><img src="{prefix}/static/loading.gif" /> {2}</div>
<div class="loaded"></div>
</div>
'''
@ -49,7 +49,7 @@ def render_book_list(ids, suffix=''): # {{{
ld = xml(json.dumps(pg), True)
rpages.append(page_template.format(i, ld,
xml(_('Loading, please wait')) + '&hellip;',
start=pos+1, end=pos+len(pg)))
start=pos+1, end=pos+len(pg), prefix=prefix))
rpages = u'\n\n'.join(rpages)
templ = u'''\
@ -91,7 +91,7 @@ def utf8(x): # {{{
return x
# }}}
def render_rating(rating, container='span', prefix=None): # {{{
def render_rating(rating, url_prefix, container='span', prefix=None): # {{{
if rating < 0.1:
return '', ''
added = 0
@ -108,15 +108,15 @@ def render_rating(rating, container='span', prefix=None): # {{{
elif n >= 0.9:
x = 'on'
ans.append(
u'<img alt="{0}" title="{0}" src="/static/star-{1}.png" />'.format(
rstring, x))
u'<img alt="{0}" title="{0}" src="{2}/static/star-{1}.png" />'.format(
rstring, x, url_prefix))
added += 1
ans.append('</%s>'%container)
return u''.join(ans), rstring
# }}}
def get_category_items(category, items, restriction, datatype): # {{{
def get_category_items(category, items, restriction, datatype, prefix): # {{{
if category == 'search':
items = [x for x in items if x.name != restriction]
@ -125,8 +125,8 @@ def get_category_items(category, items, restriction, datatype): # {{{
templ = (u'<div title="{4}" class="category-item">'
'<div class="category-name">{0}</div><div>{1}</div>'
'<div>{2}'
'<span class="href">{3}</span></div></div>')
rating, rstring = render_rating(i.avg_rating)
'<span class="href">{5}{3}</span></div></div>')
rating, rstring = render_rating(i.avg_rating, prefix)
name = xml(i.name)
if datatype == 'rating':
name = xml(_('%d stars')%int(i.avg_rating))
@ -142,7 +142,7 @@ def get_category_items(category, items, restriction, datatype): # {{{
q = category
href = '/browse/matches/%s/%s'%(quote(q), quote(id_))
return templ.format(xml(name), rating,
xml(desc), xml(href), rstring)
xml(desc), xml(href), rstring, prefix)
items = list(map(item, items))
return '\n'.join(['<div class="category-container">'] + items + ['</div>'])
@ -243,6 +243,7 @@ class BrowseServer(object):
ans = ans.replace('{sort_select_label}', xml(_('Sort by')+':'))
ans = ans.replace('{sort_cookie_name}', scn)
ans = ans.replace('{prefix}', self.opts.url_prefix)
opts = ['<option %svalue="%s">%s</option>' % (
'selected="selected" ' if k==sort else '',
xml(k), xml(n), ) for k, n in
@ -258,15 +259,14 @@ class BrowseServer(object):
ans = ans.replace('{initial_search}', initial_search)
return ans
return self.__browse_template__
@property
def browse_summary_template(self):
if not hasattr(self, '__browse_summary_template__') or \
self.opts.develop:
self.__browse_summary_template__ = \
P('content_server/browse/summary.html', data=True).decode('utf-8')
return self.__browse_summary_template__
return self.__browse_summary_template__.replace('{prefix}',
self.opts.url_prefix)
@property
def browse_details_template(self):
@ -274,7 +274,8 @@ class BrowseServer(object):
self.opts.develop:
self.__browse_details_template__ = \
P('content_server/browse/details.html', data=True).decode('utf-8')
return self.__browse_details_template__
return self.__browse_details_template__.replace('{prefix}',
self.opts.url_prefix)
# }}}
@ -334,11 +335,11 @@ class BrowseServer(object):
icon = 'blank.png'
cats.append((meta['name'], category, icon))
cats = [('<li title="{2} {0}"><img src="{src}" alt="{0}" />'
cats = [('<li title="{2} {0}"><img src="{3}{src}" alt="{0}" />'
'<span class="label">{0}</span>'
'<span class="url">/browse/category/{1}</span></li>')
'<span class="url">{3}/browse/category/{1}</span></li>')
.format(xml(x, True), xml(quote(y)), xml(_('Browse books by')),
src='/browse/icon/'+z)
self.opts.url_prefix, src='/browse/icon/'+z)
for x, y, z in cats]
main = '<div class="toplevel"><h3>{0}</h3><ul>{1}</ul></div>'\
@ -378,7 +379,8 @@ class BrowseServer(object):
if len(items) <= self.opts.max_opds_ungrouped_items:
script = 'false'
items = get_category_items(category, items,
self.search_restriction_name, datatype)
self.search_restriction_name, datatype,
self.opts.url_prefix)
else:
getter = lambda x: unicode(getattr(x, 'sort', x.name))
starts = set([])
@ -393,12 +395,13 @@ class BrowseServer(object):
getter(y).upper().startswith(x)])
items = [(u'<h3 title="{0}">{0} <span>[{2}]</span></h3><div>'
u'<div class="loaded" style="display:none"></div>'
u'<div class="loading"><img alt="{1}" src="/static/loading.gif" /><em>{1}</em></div>'
u'<span class="load_href">{3}</span></div>').format(
u'<div class="loading"><img alt="{1}" src="{4}/static/loading.gif" /><em>{1}</em></div>'
u'<span class="load_href">{4}{3}</span></div>').format(
xml(s, True),
xml(_('Loading, please wait'))+'&hellip;',
unicode(c),
xml(u'/browse/category_group/%s/%s'%(category, s)))
xml(u'/browse/category_group/%s/%s'%(category, s)),
self.opts.url_prefix)
for s, c in category_groups.items()]
items = '\n\n'.join(items)
items = u'<div id="groups">\n{0}</div>'.format(items)
@ -410,13 +413,13 @@ class BrowseServer(object):
main = u'''
<div class="category">
<h3>{0}</h3>
<a class="navlink" href="/browse"
<a class="navlink" href="{3}/browse"
title="{2}">{2}&nbsp;&uarr;</a>
{1}
</div>
'''.format(
xml(_('Browsing by')+': ' + category_name), items,
xml(_('Up'), True))
xml(_('Up'), True), self.opts.url_prefix)
return self.browse_template(sort).format(title=category_name,
script=script, main=main)
@ -449,19 +452,23 @@ class BrowseServer(object):
sort = self.browse_sort_categories(entries, sort)
entries = get_category_items(category, entries,
self.search_restriction_name, datatype)
self.search_restriction_name, datatype,
self.opts.url_prefix)
return json.dumps(entries, ensure_ascii=False)
@Endpoint()
def browse_catalog(self, category=None, category_sort=None):
'Entry point for top-level, categories and sub-categories'
prefix = '' if self.is_wsgi else self.opts.url_prefix
if category == None:
ans = self.browse_toplevel()
elif category == 'newest':
raise cherrypy.InternalRedirect('/browse/matches/newest/dummy')
raise cherrypy.InternalRedirect(prefix +
'/browse/matches/newest/dummy')
elif category == 'allbooks':
raise cherrypy.InternalRedirect('/browse/matches/allbooks/dummy')
raise cherrypy.InternalRedirect(prefix +
'/browse/matches/allbooks/dummy')
else:
ans = self.browse_category(category, category_sort)
@ -532,7 +539,8 @@ class BrowseServer(object):
list_sort = category
sort = self.browse_sort_book_list(items, list_sort)
ids = [x[0] for x in items]
html = render_book_list(ids, suffix=_('in') + ' ' + category_name)
html = render_book_list(ids, self.opts.url_prefix,
suffix=_('in') + ' ' + category_name)
return self.browse_template(sort, category=False).format(
title=_('Books in') + " " +category_name,
@ -580,17 +588,18 @@ class BrowseServer(object):
if fmts and fmt:
other_fmts = [x for x in fmts if x.lower() != fmt.lower()]
if other_fmts:
ofmts = [u'<a href="/get/{0}/{1}_{2}.{0}" title="{3}">{3}</a>'\
.format(f, fname, id_, f.upper()) for f in
ofmts = [u'<a href="{4}/get/{0}/{1}_{2}.{0}" title="{3}">{3}</a>'\
.format(f, fname, id_, f.upper(),
self.opts.url_prefix) for f in
other_fmts]
ofmts = ', '.join(ofmts)
args['other_formats'] = u'<strong>%s: </strong>' % \
_('Other formats') + ofmts
args['details_href'] = '/browse/details/'+str(id_)
args['details_href'] = self.opts.url_prefix + '/browse/details/'+str(id_)
if fmt:
href = '/get/%s/%s_%d.%s'%(
href = self.opts.url_prefix + '/get/%s/%s_%d.%s'%(
fmt, fname, id_, fmt)
rt = xml(_('Read %s in the %s format')%(args['title'],
fmt.upper()), True)
@ -603,7 +612,8 @@ class BrowseServer(object):
args['comments'] = comments_to_html(mi.comments)
args['stars'] = ''
if mi.rating:
args['stars'] = render_rating(mi.rating/2.0, prefix=_('Rating'))[0]
args['stars'] = render_rating(mi.rating/2.0,
self.opts.url_prefix, prefix=_('Rating'))[0]
if args['tags']:
args['tags'] = u'<strong>%s: </strong>'%xml(_('Tags')) + \
args['tags']
@ -628,8 +638,9 @@ class BrowseServer(object):
args, fmt, fmts, fname = self.browse_get_book_args(mi, id_)
args['formats'] = ''
if fmts:
ofmts = [u'<a href="/get/{0}/{1}_{2}.{0}" title="{3}">{3}</a>'\
.format(fmt, fname, id_, fmt.upper()) for fmt in
ofmts = [u'<a href="{4}/get/{0}/{1}_{2}.{0}" title="{3}">{3}</a>'\
.format(fmt, fname, id_, fmt.upper(),
self.opts.url_prefix) for fmt in
fmts]
ofmts = ', '.join(ofmts)
args['formats'] = ofmts
@ -648,7 +659,8 @@ class BrowseServer(object):
continue
if m['datatype'] == 'rating':
r = u'<strong>%s: </strong>'%xml(m['name']) + \
render_rating(mi.rating/2.0, prefix=m['name'])[0]
render_rating(mi.rating/2.0, self.opts.url_prefix,
prefix=m['name'])[0]
else:
r = u'<strong>%s: </strong>'%xml(m['name']) + \
args[field]
@ -704,7 +716,8 @@ class BrowseServer(object):
items = [self.db.data._data[x] for x in ids]
sort = self.browse_sort_book_list(items, list_sort)
ids = [x[0] for x in items]
html = render_book_list(ids, suffix=_('in search')+': '+query)
html = render_book_list(ids, self.opts.url_prefix,
suffix=_('in search')+': '+query)
return self.browse_template(sort, category=False, initial_search=query).format(
title=_('Matching books'),
script='booklist();', main=html)

View File

@ -103,7 +103,11 @@ class ContentServer(object):
if self.opts.develop:
lm = fromtimestamp(os.stat(path).st_mtime)
cherrypy.response.headers['Last-Modified'] = self.last_modified(lm)
return open(path, 'rb').read()
with open(path, 'rb') as f:
ans = f.read()
if path.endswith('.css'):
ans = ans.replace('/static/', self.opts.url_prefix + '/static/')
return ans
def index(self, **kwargs):
'The / URL'
@ -127,7 +131,8 @@ class ContentServer(object):
return self.browse_toplevel()
def old(self, **kwargs):
return self.static('index.html')
return self.static('index.html').replace('{prefix}',
self.opts.url_prefix)
# Actually get content from the database {{{
def get_cover(self, id, thumbnail=False, thumb_width=60, thumb_height=80):

View File

@ -24,8 +24,30 @@ def stop_threaded_server(server):
server.exit()
server.thread = None
def create_wsgi_app(path_to_library=None, prefix=''):
'WSGI entry point'
from calibre.library import db
cherrypy.config.update({'environment': 'embedded'})
db = db(path_to_library)
parser = option_parser()
opts, args = parser.parse_args(['calibre-server'])
opts.url_prefix = prefix
server = LibraryServer(db, opts, wsgi=True, show_tracebacks=True)
return cherrypy.Application(server, script_name=None, config=server.config)
def option_parser():
parser = config().option_parser('%prog '+ _('[options]\n\nStart the calibre content server.'))
parser = config().option_parser('%prog '+ _(
'''[options]
Start the calibre content server. The calibre content server
exposes your calibre library over the internet. The default interface
allows you to browse you calibre library by categories. You can also
access an interface optimized for mobile browsers at /mobile and an
OPDS based interface for use with reading applications at /opds.
The OPDS interface is advertised via BonJour automatically.
'''
))
parser.add_option('--with-library', default=None,
help=_('Path to the library folder to serve with the content server'))
parser.add_option('--pidfile', default=None,

View File

@ -26,9 +26,9 @@ def CLASS(*args, **kwargs): # class is a reserved word in Python
return kwargs
def build_search_box(num, search, sort, order): # {{{
def build_search_box(num, search, sort, order, prefix): # {{{
div = DIV(id='search_box')
form = FORM('Show ', method='get', action='mobile')
form = FORM('Show ', method='get', action=prefix+'/mobile')
form.set('accept-charset', 'UTF-8')
div.append(form)
@ -89,11 +89,12 @@ def build_navigation(start, num, total, url_base): # {{{
# }}}
def build_index(books, num, search, sort, order, start, total, url_base, CKEYS):
logo = DIV(IMG(src='/static/calibre.png', alt=__appname__), id='logo')
def build_index(books, num, search, sort, order, start, total, url_base, CKEYS,
prefix):
logo = DIV(IMG(src=prefix+'/static/calibre.png', alt=__appname__), id='logo')
search_box = build_search_box(num, search, sort, order)
navigation = build_navigation(start, num, total, url_base)
search_box = build_search_box(num, search, sort, order, prefix)
navigation = build_navigation(start, num, total, prefix+url_base)
bookt = TABLE(id='listing')
body = BODY(
@ -107,7 +108,8 @@ def build_index(books, num, search, sort, order, start, total, url_base, CKEYS):
# Book list {{{
for book in books:
thumbnail = TD(
IMG(type='image/jpeg', border='0', src='/get/thumb/%s' %
IMG(type='image/jpeg', border='0',
src=prefix+'/get/thumb/%s' %
book['id']),
CLASS('thumbnail'))
@ -118,8 +120,8 @@ def build_index(books, num, search, sort, order, start, total, url_base, CKEYS):
s = SPAN(
A(
fmt.lower(),
href='/get/%s/%s-%s_%d.%s' % (fmt, a, t,
book['id'], fmt)
href=prefix+'/get/%s/%s-%s_%d.%s' % (fmt, a, t,
book['id'], fmt.lower())
),
CLASS('button'))
s.tail = u''
@ -154,7 +156,7 @@ def build_index(books, num, search, sort, order, start, total, url_base, CKEYS):
TITLE(__appname__ + ' Library'),
LINK(rel='icon', href='http://calibre-ebook.com/favicon.ico',
type='image/x-icon'),
LINK(rel='stylesheet', type='text/css', href='/mobile/style.css')
LINK(rel='stylesheet', type='text/css', href=prefix+'/mobile/style.css')
), # End head
body
) # End html
@ -174,7 +176,9 @@ class MobileServer(object):
cherrypy.response.headers['Content-Type'] = 'text/css; charset=utf-8'
updated = utcfromtimestamp(os.stat(path).st_mtime)
cherrypy.response.headers['Last-Modified'] = self.last_modified(updated)
return open(path, 'rb').read()
with open(path, 'rb') as f:
ans = f.read()
return ans.replace('{prefix}', self.opts.url_prefix)
def mobile(self, start='1', num='25', sort='date', search='',
_=None, order='descending'):
@ -259,7 +263,8 @@ class MobileServer(object):
url_base = "/mobile?search=" + search+";order="+order+";sort="+sort+";num="+str(num)
return html.tostring(build_index(books, num, search, sort, order,
start, len(ids), url_base, CKEYS),
start, len(ids), url_base, CKEYS,
self.opts.url_prefix),
encoding='utf-8', include_meta_content_type=True,
pretty_print=True)

View File

@ -132,7 +132,7 @@ def CATALOG_GROUP_ENTRY(item, category, base_href, version, updated):
link
)
def ACQUISITION_ENTRY(item, version, db, updated, CFM, CKEYS):
def ACQUISITION_ENTRY(item, version, db, updated, CFM, CKEYS, prefix):
FM = db.FIELD_MAP
title = item[FM['title']]
if not title:
@ -185,16 +185,16 @@ def ACQUISITION_ENTRY(item, version, db, updated, CFM, CKEYS):
for fmt in formats.split(','):
fmt = fmt.lower()
mt = guess_type('a.'+fmt)[0]
href = '/get/%s/%s'%(fmt, item[FM['id']])
href = prefix + '/get/%s/%s'%(fmt, item[FM['id']])
if mt:
link = E.link(type=mt, href=href)
if version > 0:
link.set('rel', "http://opds-spec.org/acquisition")
ans.append(link)
ans.append(E.link(type='image/jpeg', href='/get/cover/%s'%item[FM['id']],
ans.append(E.link(type='image/jpeg', href=prefix+'/get/cover/%s'%item[FM['id']],
rel="x-stanza-cover-image" if version == 0 else
"http://opds-spec.org/cover"))
ans.append(E.link(type='image/jpeg', href='/get/thumb/%s'%item[FM['id']],
ans.append(E.link(type='image/jpeg', href=prefix+'/get/thumb/%s'%item[FM['id']],
rel="x-stanza-cover-image-thumbnail" if version == 0 else
"http://opds-spec.org/thumbnail"))
@ -275,7 +275,7 @@ class NavFeed(Feed):
class AcquisitionFeed(NavFeed):
def __init__(self, updated, id_, items, offsets, page_url, up_url, version,
db):
db, prefix):
NavFeed.__init__(self, id_, updated, version, offsets, page_url, up_url)
CFM = db.field_metadata
CKEYS = [key for key in sorted(custom_fields_to_display(db),
@ -283,7 +283,7 @@ class AcquisitionFeed(NavFeed):
CFM[y]['name'].lower()))]
for item in items:
self.root.append(ACQUISITION_ENTRY(item, version, db, updated,
CFM, CKEYS))
CFM, CKEYS, prefix))
class CategoryFeed(NavFeed):
@ -360,7 +360,8 @@ class OPDSServer(object):
cherrypy.response.headers['Last-Modified'] = self.last_modified(updated)
cherrypy.response.headers['Content-Type'] = 'application/atom+xml;profile=opds-catalog'
return str(AcquisitionFeed(updated, id_, items, offsets,
page_url, up_url, version, self.db))
page_url, up_url, version, self.db,
self.opts.url_prefix))
def opds_search(self, query=None, version=0, offset=0):
try:

View File

@ -463,5 +463,7 @@ Finally, you can add downloaded news to the |app| library with::
/opt/calibre/calibredb add --with-library /path/to/library outfile.epub
Remember to read the command line documentatation section of the |app| User Manual to learn more about these, and other commands.
Remember to read the command line documentation section of the |app| User Manual to learn more about these, and other commands.
.. note:: Some parts of calibre require a X server. If you're lucky, nothing you do will fall into this category, if not, you will have to look into using xvfb.

View File

@ -36,6 +36,7 @@ FileTypePlugin
.. _pluginsMetadataPlugin:
Metadata plugins
-------------------
@ -50,7 +51,6 @@ Metadata plugins
:members:
:member-order: bysource
.. _pluginsMetadataSource:
Catalog plugins
----------------
@ -60,6 +60,7 @@ Catalog plugins
:members:
:member-order: bysource
.. _pluginsMetadataSource:
Metadata download plugins
--------------------------

View File

@ -0,0 +1,108 @@
.. include:: global.rst
.. _servertutorial:
Integrating the |app| content server into other servers
==========================================================
Here, we will show you how to integrate the |app| content server into another server. The most common reason for this is to make use of SSL or more sophisticated authentication. There are two main techniques: Running the |app| content server as a standalone process and using a reverse proxy to connect it with your main server or running the content server in process in your main server with WSGI. The examples below are all for Apache 2.x on linux, but should be easily adaptable to other platforms.
.. contents:: Contents
:depth: 2
:local:
.. note:: This only applies to calibre releases >= 0.7.25
Using a reverse proxy
-----------------------
This is the simplest approach as it allows you to use the binary calibre install with no external dependencies/system integration requirements.
First start the |app| content server as shown below::
calibre-server --url-prefix /calibre --port 8080
Now suppose you are using Apache as your main server. First enable the proxy modules in apache, by adding the following to :file:`httpd.conf`::
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
The exact technique for enabling the proxy modules will vary depending on your Apache installation. Once you have the proxy modules enabled, add the following rules to httpd.conf (or if you are using virtual hosts to the conf file for the virtual host in question::
RewriteEngine on
RewriteRule ^/calibre/(.*) http://localhost:8080/calibre/$1 [proxy]
RewriteRule ^/calibre http://localhost:8080 [proxy]
That's all, you will now be able to access the |app| Content Server under the /calibre URL in your apache server.
.. note:: If you are willing to devote an entire VirtualHost to the content server, then there is no need to use --url-prefix and RewriteRule, instead just use the ProxyPass directive.
In process
------------
The calibre content server can be run directly, in process, inside a host server like Apache using the WSGI framework.
.. note:: For this to work, all the dependencies needed by calibre must be installed on your system. On linux, this can be achieved fairly easily by installing the distribution provided calibre package (provided it is up to date).
First, we have to create a WSGI *adapter* for the calibre content server. Here is a template you can use for the purpose. Replace the paths as directed in the comments
.. code-block:: python
# WSGI script file to run calibre content server as a WSGI app
import sys, os
# You can get the paths referenced here by running
# calibre-debug --paths
# on your server
# The first entry from CALIBRE_PYTHON_PATH
sys.path.insert(0, '/home/kovid/work/calibre/src')
# CALIBRE_RESOURCES_PATH
sys.resources_location = '/home/kovid/work/calibre/resources'
# CALIBRE_EXTENSIONS_PATH
sys.extensions_location = '/home/kovid/work/calibre/src/calibre/plugins'
# Path to directory containing calibre executables
sys.executables_location = '/usr/bin'
# Path to a directory for which the server has read/write permissions
# calibre config will be stored here
os.environ['CALIBRE_CONFIG_DIRECTORY'] = '/var/www/localhost/calibre-config'
del sys
del os
from calibre.library.server.main import create_wsgi_app
application = create_wsgi_app(
# The mount point of this WSGI application (i.e. the first argument to
# the WSGIScriptAlias directive). Set to empty string is mounted at /
prefix='/calibre',
# Path to the calibre library to be served
# The server process must have write permission for all files/dirs
# in this directory or BAD things will happen
path_to_library='/home/kovid/documents/demo library'
)
del create_wsgi_app
Save this adapter as :file:`calibre-wsgi-adpater.py` somewhere your server will have access to it.
Let's suppose that we want to use WSGI in Apache. First enable WSGI in Apache by adding the following to :file:`httpd.conf`::
LoadModule proxy_module modules/mod_wsgi.so
The exact technique for enabling the wsgi module will vary depending on your Apache installation. Once you have the proxy modules enabled, add the following rules to httpd.conf (or if you are using virtual hosts to the conf file for the virtual host in question::
WSGIScriptAlias /calibre /var/www/localhost/cgi-bin/calibre-wsgi-adapter.py
Change the path to :file:`calibre-wsgi-adapter.py` to wherever you saved it previously (make sure Apache has access to it).
That's all, you will now be able to access the |app| Content Server under the /calibre URL in your apache server.
.. note:: For more help with using mod_wsgi in Apache, see `mod_wsgi <http://code.google.com/p/modwsgi/wiki/WhereToGetHelp>`_.

View File

@ -16,4 +16,5 @@ Here you will find tutorials to get you started using |app|'s more advanced feat
template_lang
regexp
portable
server

View File

@ -7,7 +7,7 @@ being closed.
"""
import tempfile, os, atexit, binascii, cPickle
from calibre import __version__, __appname__
from calibre.constants import __version__, __appname__
def cleanup(path):
try:

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -583,7 +583,7 @@ class BasicNewsRecipe(Recipe):
self.title = unicode(self.title, 'utf-8', 'replace')
self.debug = options.verbose > 1
self.output_dir = os.getcwd()
self.output_dir = os.path.abspath(os.getcwdu())
self.verbose = options.verbose
self.test = options.test
self.username = options.username
@ -594,7 +594,6 @@ class BasicNewsRecipe(Recipe):
if self.touchscreen:
self.template_css += self.output_profile.touchscreen_news_css
self.output_dir = os.path.abspath(self.output_dir)
if options.test:
self.max_articles_per_feed = 2
self.simultaneous_downloads = min(4, self.simultaneous_downloads)
@ -958,6 +957,8 @@ class BasicNewsRecipe(Recipe):
self.log.error(_('Could not download cover: %s')%str(err))
self.log.debug(traceback.format_exc())
else:
if not cu:
return
cdata = None
if os.access(cu, os.R_OK):
cdata = open(cu, 'rb').read()
@ -988,6 +989,7 @@ class BasicNewsRecipe(Recipe):
self.cover_path = cpath
def download_cover(self):
self.cover_path = None
try:
self._download_cover()
except: