Cutlery/calibre
1
0
Fork 0
mirror of https://github.com/kovidgoyal/calibre.git synced 2025-07-09 03:04:10 -04:00
Code Issues Packages Projects Releases Wiki Activity

Merge from trunk

Browse Source
This commit is contained in:
Charles Haley 2013-01-02 11:22:27 +01:00
commit 05887d353f
36 changed files with 1858 additions and 1079 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
README
View File

@ -1,7 +1,7 @@
calibre is an e-book library manager. It can view, convert and catalog e-books \ calibre is an e-book library manager. It can view, convert and catalog e-books
in most of the major e-book formats. It can also talk to e-book reader \ in most of the major e-book formats. It can also talk to e-book reader
devices. It can go out to the internet and fetch metadata for your books. \ devices. It can go out to the internet and fetch metadata for your books.
It can download newspapers and convert them into e-books for convenient \ It can download newspapers and convert them into e-books for convenient
reading. It is cross platform, running on Linux, Windows and OS X. reading. It is cross platform, running on Linux, Windows and OS X.
For screenshots: https://calibre-ebook.com/demo For screenshots: https://calibre-ebook.com/demo
@ -15,5 +15,5 @@ bzr branch lp:calibre
To update your copy of the source code: To update your copy of the source code:
bzr merge bzr merge
Tarballs of the source code for each release are now available \ Tarballs of the source code for each release are now available
at http://code.google.com/p/calibre-ebook at http://code.google.com/p/calibre-ebook

6
manual/faq.rst
View File

@ -162,7 +162,8 @@ Follow these steps to find the problem:
* If you are connecting an Apple iDevice (iPad, iPod Touch, iPhone), use the 'Connect to iTunes' method in the 'Getting started' instructions in `Calibre + Apple iDevices: Start here <http://www.mobileread.com/forums/showthread.php?t=118559>`_. * If you are connecting an Apple iDevice (iPad, iPod Touch, iPhone), use the 'Connect to iTunes' method in the 'Getting started' instructions in `Calibre + Apple iDevices: Start here <http://www.mobileread.com/forums/showthread.php?t=118559>`_.
* Make sure you are running the latest version of |app|. The latest version can always be downloaded from `the calibre website <http://calibre-ebook.com/download>`_. * Make sure you are running the latest version of |app|. The latest version can always be downloaded from `the calibre website <http://calibre-ebook.com/download>`_.
* Ensure your operating system is seeing the device. That is, the device should show up in Windows Explorer (in Windows) or Finder (in OS X). * Ensure your operating system is seeing the device. That is, the device should show up in Windows Explorer (in Windows) or Finder (in OS X).
* In |app|, go to Preferences->Plugins->Device Interface plugin and make sure the plugin for your device is enabled, the plugin icon next to it should be green when it is enabled. * In |app|, go to Preferences->Ignored Devices and check that your device
is not being ignored
* If all the above steps fail, go to Preferences->Miscellaneous and click debug device detection with your device attached and post the output as a ticket on `the calibre bug tracker <http://bugs.calibre-ebook.com>`_. * If all the above steps fail, go to Preferences->Miscellaneous and click debug device detection with your device attached and post the output as a ticket on `the calibre bug tracker <http://bugs.calibre-ebook.com>`_.
My device is non-standard or unusual. What can I do to connect to it? My device is non-standard or unusual. What can I do to connect to it?
@ -668,6 +669,9 @@ There are three possible things I know of, that can cause this:
the blacklist of programs inside RoboForm to fix this. Or uninstall the blacklist of programs inside RoboForm to fix this. Or uninstall
RoboForm. RoboForm.
* The Logitech SetPoint Settings application causes random crashes in
|app| when it is open. Close it before starting |app|.
|app| is not starting on OS X? |app| is not starting on OS X?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

8
recipes/alternet.recipe
View File

@ -10,14 +10,12 @@ class Alternet(BasicNewsRecipe):
category = 'News, Magazine' category = 'News, Magazine'
description = 'News magazine and online community' description = 'News magazine and online community'
feeds = [ feeds = [
(u'Front Page', u'http://feeds.feedblitz.com/alternet'), (u'Front Page', u'http://feeds.feedblitz.com/alternet')
(u'Breaking News', u'http://feeds.feedblitz.com/alternet_breaking_news'),
(u'Top Ten Campaigns', u'http://feeds.feedblitz.com/alternet_top_10_campaigns'),
(u'Special Coverage Areas', u'http://feeds.feedblitz.com/alternet_coverage')
] ]
remove_attributes = ['width', 'align','cellspacing'] remove_attributes = ['width', 'align','cellspacing']
remove_javascript = True remove_javascript = True
use_embedded_content = False use_embedded_content = True
no_stylesheets = True no_stylesheets = True
language = 'en' language = 'en'
encoding = 'UTF-8' encoding = 'UTF-8'

88
recipes/foreignaffairs.recipe
View File

@ -11,21 +11,21 @@ class ForeignAffairsRecipe(BasicNewsRecipe):
by Chen Wei weichen302@gmx.com, 2012-02-05''' by Chen Wei weichen302@gmx.com, 2012-02-05'''
__license__ = 'GPL v3' __license__ = 'GPL v3'
__author__ = 'kwetal' __author__ = 'Rick Shang, kwetal'
language = 'en' language = 'en'
version = 1.01 version = 1.01
title = u'Foreign Affairs (Subcription or (free) Registration)' title = u'Foreign Affairs (Subcription)'
publisher = u'Council on Foreign Relations' publisher = u'Council on Foreign Relations'
category = u'USA, Foreign Affairs' category = u'USA, Foreign Affairs'
description = u'The leading forum for serious discussion of American foreign policy and international affairs.' description = u'The leading forum for serious discussion of American foreign policy and international affairs.'
no_stylesheets = True no_stylesheets = True
remove_javascript = True remove_javascript = True
needs_subscription = True
INDEX = 'http://www.foreignaffairs.com' INDEX = 'http://www.foreignaffairs.com'
FRONTPAGE = 'http://www.foreignaffairs.com/magazine' FRONTPAGE = 'http://www.foreignaffairs.com/magazine'
INCLUDE_PREMIUM = False
remove_tags = [] remove_tags = []
@ -68,42 +68,56 @@ class ForeignAffairsRecipe(BasicNewsRecipe):
def parse_index(self): def parse_index(self):
answer = [] answer = []
soup = self.index_to_soup(self.FRONTPAGE) soup = self.index_to_soup(self.FRONTPAGE)
sec_start = soup.findAll('div', attrs={'class':'panel-separator'}) #get dates
date = re.split('\s\|\s',self.tag_to_string(soup.head.title.string))[0]
self.timefmt = u' [%s]'%date
sec_start = soup.findAll('div', attrs= {'class':'panel-pane'})
for sec in sec_start: for sec in sec_start:
content = sec.nextSibling
if content:
section = self.tag_to_string(content.find('h2'))
articles = [] articles = []
section = self.tag_to_string(sec.find('h2'))
tags = [] if 'Books' in section:
for div in content.findAll('div', attrs = {'class': re.compile(r'view-row\s+views-row-[0-9]+\s+views-row-[odd|even].*')}): reviewsection=sec.find('div', attrs = {'class': 'item-list'})
tags.append(div) for subsection in reviewsection.findAll('div'):
for li in content.findAll('li'): subsectiontitle=self.tag_to_string(subsection.span.a)
tags.append(li) subsectionurl=self.INDEX + subsection.span.a['href']
soup1 = self.index_to_soup(subsectionurl)
for div in tags: for div in soup1.findAll('div', attrs = {'class': 'views-field-title'}):
title = url = description = author = None if div.find('a') is not None:
originalauthor=self.tag_to_string(div.findNext('div', attrs = {'class':'views-field-field-article-book-nid'}).div.a)
if self.INCLUDE_PREMIUM: title=subsectiontitle+': '+self.tag_to_string(div.span.a)+' by '+originalauthor
found_premium = False url=self.INDEX+div.span.a['href']
atr=div.findNext('div', attrs = {'class': 'views-field-field-article-display-authors-value'})
if atr is not None:
author=self.tag_to_string(atr.span.a)
else: else:
found_premium = div.findAll('span', attrs={'class': author=''
'premium-icon'}) desc=div.findNext('span', attrs = {'class': 'views-field-field-article-summary-value'})
if not found_premium: if desc is not None:
tag = div.find('div', attrs={'class': 'views-field-title'}) description=self.tag_to_string(desc.div.p)
else:
if tag: description=''
a = tag.find('a') articles.append({'title':title, 'date':None, 'url':url, 'description':description, 'author':author})
if a: subsectiontitle=''
title = self.tag_to_string(a) else:
url = self.INDEX + a['href'] for div in sec.findAll('div', attrs = {'class': 'views-field-title'}):
author = self.tag_to_string(div.find('div', attrs = {'class': 'views-field-field-article-display-authors-value'})) if div.find('a') is not None:
tag_summary = div.find('span', attrs = {'class': 'views-field-field-article-summary-value'}) title=self.tag_to_string(div.span.a)
description = self.tag_to_string(tag_summary) url=self.INDEX+div.span.a['href']
articles.append({'title':title, 'date':None, 'url':url, atr=div.findNext('div', attrs = {'class': 'views-field-field-article-display-authors-value'})
'description':description, 'author':author}) if atr is not None:
author=self.tag_to_string(atr.span.a)
else:
author=''
desc=div.findNext('span', attrs = {'class': 'views-field-field-article-summary-value'})
if desc is not None:
description=self.tag_to_string(desc.div.p)
else:
description=''
articles.append({'title':title, 'date':None, 'url':url, 'description':description, 'author':author})
if articles: if articles:
answer.append((section, articles)) answer.append((section, articles))
return answer return answer
@ -115,15 +129,17 @@ class ForeignAffairsRecipe(BasicNewsRecipe):
return soup return soup
needs_subscription = True
def get_browser(self): def get_browser(self):
br = BasicNewsRecipe.get_browser() br = BasicNewsRecipe.get_browser()
if self.username is not None and self.password is not None: if self.username is not None and self.password is not None:
br.open('https://www.foreignaffairs.com/user?destination=home') br.open('https://www.foreignaffairs.com/user?destination=user%3Fop%3Dlo')
br.select_form(nr = 1) br.select_form(nr = 1)
br['name'] = self.username br['name'] = self.username
br['pass'] = self.password br['pass'] = self.password
br.submit() br.submit()
return br return br
def cleanup(self):
self.browser.open('http://www.foreignaffairs.com/logout?destination=user%3Fop=lo')

35
recipes/harpers_full.recipe
View File

@ -8,7 +8,7 @@ If you have institutional subscription based on access IP you do not need to ent
anything in username/password fields anything in username/password fields
''' '''
import time import time, re
import urllib import urllib
from calibre import strftime from calibre import strftime
from calibre.web.feeds.news import BasicNewsRecipe from calibre.web.feeds.news import BasicNewsRecipe
@ -29,7 +29,6 @@ class Harpers_full(BasicNewsRecipe):
needs_subscription = 'optional' needs_subscription = 'optional'
masthead_url = 'http://harpers.org/wp-content/themes/harpers/images/pheader.gif' masthead_url = 'http://harpers.org/wp-content/themes/harpers/images/pheader.gif'
publication_type = 'magazine' publication_type = 'magazine'
INDEX = strftime('http://harpers.org/archive/%Y/%m')
LOGIN = 'http://harpers.org/wp-content/themes/harpers/ajax_login.php' LOGIN = 'http://harpers.org/wp-content/themes/harpers/ajax_login.php'
extra_css = """ extra_css = """
body{font-family: adobe-caslon-pro,serif} body{font-family: adobe-caslon-pro,serif}
@ -65,17 +64,28 @@ class Harpers_full(BasicNewsRecipe):
return br return br
def parse_index(self): def parse_index(self):
#find current issue
soup = self.index_to_soup('http://harpers.org/')
currentIssue=soup.find('div',attrs={'class':'mainNavi'}).find('li',attrs={'class':'curentIssue'})
currentIssue_url=self.tag_to_string(currentIssue.a['href'])
self.log(currentIssue_url)
#go to the current issue
soup1 = self.index_to_soup(currentIssue_url)
date = re.split('\s\|\s',self.tag_to_string(soup1.head.title.string))[0]
self.timefmt = u' [%s]'%date
#get cover
coverurl='http://harpers.org/wp-content/themes/harpers/ajax_microfiche.php?img=harpers-'+re.split('harpers.org/',currentIssue_url)[1]+'gif/0001.gif'
soup2 = self.index_to_soup(coverurl)
self.cover_url = self.tag_to_string(soup2.find('img')['src'])
self.log(self.cover_url)
articles = [] articles = []
print 'Processing ' + self.INDEX
soup = self.index_to_soup(self.INDEX)
count = 0 count = 0
for item in soup.findAll('div', attrs={'class':'articleData'}): for item in soup1.findAll('div', attrs={'class':'articleData'}):
text_links = item.findAll('h2') text_links = item.findAll('h2')
for text_link in text_links: for text_link in text_links:
if count == 0: if count == 0:
lcover_url = item.find(attrs={'class':'dwpdf'})
if lcover_url:
self.cover_url = lcover_url.a['href']
count = 1 count = 1
else: else:
url = text_link.a['href'] url = text_link.a['href']
@ -87,7 +97,14 @@ class Harpers_full(BasicNewsRecipe):
,'url' :url ,'url' :url
,'description':'' ,'description':''
}) })
return [(soup.head.title.string, articles)] return [(soup1.head.title.string, articles)]
def print_version(self, url): def print_version(self, url):
return url + '?single=1' return url + '?single=1'
def cleanup(self):
soup = self.index_to_soup('http://harpers.org/')
signouturl=self.tag_to_string(soup.find('li', attrs={'class':'subLogOut'}).findNext('li').a['href'])
self.log(signouturl)
self.browser.open(signouturl)

65
recipes/la_stampa.recipe
View File

@ -2,7 +2,7 @@
__license__ = 'GPL v3' __license__ = 'GPL v3'
__author__ = 'Gabriele Marini, based on Darko Miletic' __author__ = 'Gabriele Marini, based on Darko Miletic'
__copyright__ = '2009, Darko Miletic <darko.miletic at gmail.com>' __copyright__ = '2009, Darko Miletic <darko.miletic at gmail.com>'
__description__ = 'La Stampa 05/05/2010' __description__ = 'La Stampa 28/12/2012'
''' '''
http://www.lastampa.it/ http://www.lastampa.it/
@ -14,10 +14,11 @@ class LaStampa(BasicNewsRecipe):
title = u'La Stampa' title = u'La Stampa'
language = 'it' language = 'it'
__author__ = 'Gabriele Marini' __author__ = 'Gabriele Marini'
oldest_article = 15 #oldest_article = 15
oldest_articlce = 7 #for daily schedule
max_articles_per_feed = 50 max_articles_per_feed = 50
recursion = 100 recursion = 100
cover_url = 'http://www.lastampa.it/edicola/PDF/1.pdf' cover_url = 'http://www1.lastampa.it/edicola/PDF/1.pdf'
use_embedded_content = False use_embedded_content = False
remove_javascript = True remove_javascript = True
no_stylesheets = True no_stylesheets = True
@ -33,35 +34,41 @@ class LaStampa(BasicNewsRecipe):
if link: if link:
return link[0]['href'] return link[0]['href']
keep_only_tags = [dict(attrs={'class':['boxocchiello2','titoloRub','titologir','catenaccio','sezione','articologirata']}), keep_only_tags = [dict(attrs={'class':['boxocchiello2','titoloRub','titologir','autore-girata','luogo-girata','catenaccio','sezione','articologirata','bodytext','news-single-img','ls-articoloCorpo','ls-blog-list-1col']}),
dict(name='div', attrs={'id':'corpoarticolo'}) dict(name='div', attrs={'id':'corpoarticolo'})
] ]
remove_tags = [dict(name='div', attrs={'id':'menutop'}),
dict(name='div', attrs={'id':'fwnetblocco'}), remove_tags = [dict(name='div', attrs={'id':['menutop','fwnetblocco']}),
dict(name='table', attrs={'id':'strumenti'}), dict(attrs={'class':['ls-toolbarCommenti','ls-boxCommentsBlog']}),
dict(name='table', attrs={'id':'imgesterna'}), dict(name='table', attrs={'id':['strumenti','imgesterna']}),
dict(name='a', attrs={'class':'linkblu'}), dict(name='a', attrs={'class':['linkblu','link']}),
dict(name='a', attrs={'class':'link'}),
dict(name='span', attrs={'class':['boxocchiello','boxocchiello2','sezione']}) dict(name='span', attrs={'class':['boxocchiello','boxocchiello2','sezione']})
] ]
feeds = [(u'BuonGiorno',u'http://www.lastampa.it/cultura/opinioni/buongiorno/rss.xml'),
feeds = [ (u'Jena', u'http://www.lastampa.it/cultura/opinioni/jena/rss.xml'),
(u'Home', u'http://www.lastampa.it/redazione/rss_home.xml'), (u'Editoriali', u'http://www.lastampa.it/cultura/opinioni/editoriali'),
(u'Editoriali', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=25'), (u'Finestra sull America', u'http://lastampa.feedsportal.com/c/32418/f/625713/index.rss'),
(u'Politica', u'http://www.lastampa.it/redazione/cmssezioni/politica/rss_politica.xml'), (u'HomePage', u'http://www.lastampa.it/rss.xml'),
(u'ArciItaliana', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=14'), (u'Politica Italia', u'http://www.lastampa.it/italia/politica/rss.xml'),
(u'Cronache', u'http://www.lastampa.it/redazione/cmssezioni/cronache/rss_cronache.xml'), (u'ArciItaliana', u'http://www.lastampa.it/rss/blog/arcitaliana'),
(u'Esteri', u'http://www.lastampa.it/redazione/cmssezioni/esteri/rss_esteri.xml'), (u'Cronache', u'http://www.lastampa.it/italia/cronache/rss.xml'),
(u'Danni Collaterali', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=90'), (u'Esteri', u'http://www.lastampa.it/esteri/rss.xml'),
(u'Economia', u'http://www.lastampa.it/redazione/cmssezioni/economia/rss_economia.xml'), (u'Danni Collaterali', u'http://www.lastampa.it/rss/blog/danni-collaterali'),
(u'Tecnologia ', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=30'), (u'Economia', u'http://www.lastampa.it/economia/rss.xml'),
(u'Spettacoli', u'http://www.lastampa.it/redazione/cmssezioni/spettacoli/rss_spettacoli.xml'), (u'Tecnologia ', u'http://www.lastampa.it/tecnologia/rss.xml'),
(u'Sport', u'http://www.lastampa.it/sport/rss_home.xml'), (u'Spettacoli', u'http://www.lastampa.it/spettacoli/rss.xml'),
(u'Torino', u'http://rss.feedsportal.com/c/32418/f/466938/index.rss'), (u'Sport', u'http://www.lastampa.it/sport/rss.xml'),
(u'Motori', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=57'), (u'Torino', u'http://www.lastampa.it/cronaca/rss.xml'),
(u'Scienza', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=38'), (u'Motori', u'http://www.lastampa.it/motori/rss.xml'),
(u'Fotografia', u'http://rss.feedsportal.com/c/32418/f/478449/index.rss'), (u'Scienza', u'http://www.lastampa.it/scienza/rss.xml'),
(u'Scuola', u'http://www.lastampa.it/cmstp/rubriche/oggetti/rss.asp?ID_blog=60'), (u'Cultura', u'http://www.lastampa.it/cultura/rss.xml'),
(u'Tempo Libero', u'http://www.lastampa.it/tempolibero/rss_home.xml') (u'Scuola', u'http://www.lastampa.it/cultura/scuola/rss.xml'),
(u'Benessere', u'http://www.lastampa.it/scienza/benessere/rss.xml'),
(u'Cucina', u'http://www.lastampa.it/societa/cucina/rss.xml'),
(u'Casa', u'http://www.lastampa.it/societa/casa/rss.xml'),
(u'Moda',u'http://www.lastampa.it/societa/moda/rss.xml'),
(u'Giochi',u'http://www.lastampa.it/tecnologia/giochi/rss.xml'),
(u'Viaggi',u'http://www.lastampa.it/societa/viaggi/rss.xml'),
(u'Ambiente', u'http://www.lastampa.it/scienza/ambiente/rss.xml')
] ]

118
recipes/nytimes.recipe
View File

@ -15,6 +15,7 @@ from calibre.ebooks.BeautifulSoup import BeautifulSoup, Tag, BeautifulStoneSoup
class NYTimes(BasicNewsRecipe): class NYTimes(BasicNewsRecipe):
recursions=1 # set this to zero to omit Related articles lists recursions=1 # set this to zero to omit Related articles lists
match_regexps=[r'/[12][0-9][0-9][0-9]/[0-9]+/'] # speeds up processing by preventing index page links from being followed
# set getTechBlogs to True to include the technology blogs # set getTechBlogs to True to include the technology blogs
# set tech_oldest_article to control article age # set tech_oldest_article to control article age
@ -24,6 +25,14 @@ class NYTimes(BasicNewsRecipe):
tech_oldest_article = 14 tech_oldest_article = 14
tech_max_articles_per_feed = 25 tech_max_articles_per_feed = 25
# set getPopularArticles to False if you don't want the Most E-mailed and Most Viewed articles
# otherwise you will get up to 20 of the most popular e-mailed and viewed articles (in each category)
getPopularArticles = True
popularPeriod = '1' # set this to the number of days to include in the measurement
# e.g. 7 will get the most popular measured over the last 7 days
# and 30 will get the most popular measured over 30 days.
# you still only get up to 20 articles in each category
# set headlinesOnly to True for the headlines-only version. If True, webEdition is ignored. # set headlinesOnly to True for the headlines-only version. If True, webEdition is ignored.
headlinesOnly = True headlinesOnly = True
@ -153,7 +162,7 @@ class NYTimes(BasicNewsRecipe):
timefmt = '' timefmt = ''
simultaneous_downloads = 1 #simultaneous_downloads = 1 # no longer required to deal with ads
cover_margins = (18,18,'grey99') cover_margins = (18,18,'grey99')
@ -204,7 +213,8 @@ class NYTimes(BasicNewsRecipe):
re.compile('^subNavigation'), re.compile('^subNavigation'),
re.compile('^leaderboard'), re.compile('^leaderboard'),
re.compile('^module'), re.compile('^module'),
re.compile('commentCount') re.compile('commentCount'),
'credit'
]}), ]}),
dict(name='div', attrs={'class':re.compile('toolsList')}), # bits dict(name='div', attrs={'class':re.compile('toolsList')}), # bits
dict(name='div', attrs={'class':re.compile('postNavigation')}), # bits dict(name='div', attrs={'class':re.compile('postNavigation')}), # bits
@ -291,11 +301,11 @@ class NYTimes(BasicNewsRecipe):
del ans[idx] del ans[idx]
idx_max = idx_max-1 idx_max = idx_max-1
continue continue
if self.verbose: if True: #self.verbose
self.log("Section %s: %d articles" % (ans[idx][0], len(ans[idx][1])) ) self.log("Section %s: %d articles" % (ans[idx][0], len(ans[idx][1])) )
for article in ans[idx][1]: for article in ans[idx][1]:
total_article_count += 1 total_article_count += 1
if self.verbose: if True: #self.verbose
self.log("\t%-40.40s... \t%-60.60s..." % (article['title'].encode('cp1252','replace'), self.log("\t%-40.40s... \t%-60.60s..." % (article['title'].encode('cp1252','replace'),
article['url'].encode('cp1252','replace'))) article['url'].encode('cp1252','replace')))
idx = idx+1 idx = idx+1
@ -351,23 +361,8 @@ class NYTimes(BasicNewsRecipe):
br = BasicNewsRecipe.get_browser() br = BasicNewsRecipe.get_browser()
return br return br
## This doesn't work (and probably never did). It either gets another serve of the advertisement,
## or if it gets the article then get_soup (from which it is invoked) traps trying to do xml decoding.
##
## def skip_ad_pages(self, soup):
## # Skip ad pages served before actual article
## skip_tag = soup.find(True, {'name':'skip'})
## if skip_tag is not None:
## self.log.warn("Found forwarding link: %s" % skip_tag.parent['href'])
## url = 'http://www.nytimes.com' + re.sub(r'\?.*', '', skip_tag.parent['href'])
## url += '?pagewanted=all'
## self.log.warn("Skipping ad to article at '%s'" % url)
## return self.index_to_soup(url, raw=True)
cover_tag = 'NY_NYT' cover_tag = 'NY_NYT'
def get_cover_url(self): def get_cover_url(self):
from datetime import timedelta, date
cover = 'http://webmedia.newseum.org/newseum-multimedia/dfp/jpg'+str(date.today().day)+'/lg/'+self.cover_tag+'.jpg' cover = 'http://webmedia.newseum.org/newseum-multimedia/dfp/jpg'+str(date.today().day)+'/lg/'+self.cover_tag+'.jpg'
br = BasicNewsRecipe.get_browser() br = BasicNewsRecipe.get_browser()
daysback=1 daysback=1
@ -390,6 +385,7 @@ class NYTimes(BasicNewsRecipe):
masthead_url = 'http://graphics8.nytimes.com/images/misc/nytlogo379x64.gif' masthead_url = 'http://graphics8.nytimes.com/images/misc/nytlogo379x64.gif'
def short_title(self): def short_title(self):
return self.title return self.title
@ -398,6 +394,7 @@ class NYTimes(BasicNewsRecipe):
from contextlib import closing from contextlib import closing
import copy import copy
from calibre.ebooks.chardet import xml_to_unicode from calibre.ebooks.chardet import xml_to_unicode
print("ARTICLE_TO_SOUP "+url_or_raw)
if re.match(r'\w+://', url_or_raw): if re.match(r'\w+://', url_or_raw):
br = self.clone_browser(self.browser) br = self.clone_browser(self.browser)
open_func = getattr(br, 'open_novisit', br.open) open_func = getattr(br, 'open_novisit', br.open)
@ -489,6 +486,67 @@ class NYTimes(BasicNewsRecipe):
description=description, author=author, description=description, author=author,
content='')) content=''))
def get_popular_articles(self,ans):
if self.getPopularArticles:
popular_articles = {}
key_list = []
def handleh3(h3tag):
try:
url = h3tag.a['href']
except:
return ('','','','')
url = re.sub(r'\?.*', '', url)
if self.exclude_url(url):
return ('','','','')
url += '?pagewanted=all'
title = self.tag_to_string(h3tag.a,False)
h6tag = h3tag.findNextSibling('h6')
if h6tag is not None:
author = self.tag_to_string(h6tag,False)
else:
author = ''
ptag = h3tag.findNextSibling('p')
if ptag is not None:
desc = self.tag_to_string(ptag,False)
else:
desc = ''
return(title,url,author,desc)
have_emailed = False
emailed_soup = self.index_to_soup('http://www.nytimes.com/most-popular-emailed?period='+self.popularPeriod)
for h3tag in emailed_soup.findAll('h3'):
(title,url,author,desc) = handleh3(h3tag)
if url=='':
continue
if not have_emailed:
key_list.append('Most E-Mailed')
popular_articles['Most E-Mailed'] = []
have_emailed = True
popular_articles['Most E-Mailed'].append(
dict(title=title, url=url, date=strftime('%a, %d %b'),
description=desc, author=author,
content=''))
have_viewed = False
viewed_soup = self.index_to_soup('http://www.nytimes.com/most-popular-viewed?period='+self.popularPeriod)
for h3tag in viewed_soup.findAll('h3'):
(title,url,author,desc) = handleh3(h3tag)
if url=='':
continue
if not have_viewed:
key_list.append('Most Viewed')
popular_articles['Most Viewed'] = []
have_viewed = True
popular_articles['Most Viewed'].append(
dict(title=title, url=url, date=strftime('%a, %d %b'),
description=desc, author=author,
content=''))
viewed_ans = [(k, popular_articles[k]) for k in key_list if popular_articles.has_key(k)]
for x in viewed_ans:
ans.append(x)
return ans
def get_tech_feeds(self,ans): def get_tech_feeds(self,ans):
if self.getTechBlogs: if self.getTechBlogs:
tech_articles = {} tech_articles = {}
@ -550,7 +608,7 @@ class NYTimes(BasicNewsRecipe):
self.handle_article(lidiv) self.handle_article(lidiv)
self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)] self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)]
return self.filter_ans(self.get_tech_feeds(self.ans)) return self.filter_ans(self.get_tech_feeds(self.get_popular_articles(self.ans)))
def parse_todays_index(self): def parse_todays_index(self):
@ -583,7 +641,7 @@ class NYTimes(BasicNewsRecipe):
self.handle_article(lidiv) self.handle_article(lidiv)
self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)] self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)]
return self.filter_ans(self.get_tech_feeds(self.ans)) return self.filter_ans(self.get_tech_feeds(self.get_popular_articles(self.ans)))
def parse_headline_index(self): def parse_headline_index(self):
@ -657,7 +715,7 @@ class NYTimes(BasicNewsRecipe):
self.articles[section_name].append(dict(title=title, url=url, date=pubdate, description=description, author=author, content='')) self.articles[section_name].append(dict(title=title, url=url, date=pubdate, description=description, author=author, content=''))
self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)] self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)]
return self.filter_ans(self.get_tech_feeds(self.ans)) return self.filter_ans(self.get_tech_feeds(self.get_popular_articles(self.ans)))
def parse_index(self): def parse_index(self):
if self.headlinesOnly: if self.headlinesOnly:
@ -745,11 +803,12 @@ class NYTimes(BasicNewsRecipe):
def preprocess_html(self, soup): def preprocess_html(self, soup):
print("PREPROCESS TITLE="+self.tag_to_string(soup.title)) #print(strftime("%H:%M:%S")+" -- PREPROCESS TITLE="+self.tag_to_string(soup.title))
skip_tag = soup.find(True, {'name':'skip'}) skip_tag = soup.find(True, {'name':'skip'})
if skip_tag is not None: if skip_tag is not None:
url = 'http://www.nytimes.com' + re.sub(r'\?.*', '', skip_tag.parent['href']) #url = 'http://www.nytimes.com' + re.sub(r'\?.*', '', skip_tag.parent['href'])
url += '?pagewanted=all' url = 'http://www.nytimes.com' + skip_tag.parent['href']
#url += '?pagewanted=all'
self.log.warn("Skipping ad to article at '%s'" % url) self.log.warn("Skipping ad to article at '%s'" % url)
sleep(5) sleep(5)
soup = self.handle_tags(self.article_to_soup(url)) soup = self.handle_tags(self.article_to_soup(url))
@ -920,6 +979,7 @@ class NYTimes(BasicNewsRecipe):
for aside in soup.findAll('div','aside'): for aside in soup.findAll('div','aside'):
aside.extract() aside.extract()
soup = self.strip_anchors(soup,True) soup = self.strip_anchors(soup,True)
#print("RECURSIVE: "+self.tag_to_string(soup.title))
if soup.find('div',attrs={'id':'blogcontent'}) is None: if soup.find('div',attrs={'id':'blogcontent'}) is None:
if first_fetch: if first_fetch:
@ -1005,7 +1065,7 @@ class NYTimes(BasicNewsRecipe):
if headline: if headline:
tag = Tag(soup, "h2") tag = Tag(soup, "h2")
tag['class'] = "headline" tag['class'] = "headline"
tag.insert(0, self.fixChars(headline.renderContents())) tag.insert(0, self.fixChars(self.tag_to_string(headline,False)))
soup.insert(0, tag) soup.insert(0, tag)
hrs = soup.findAll('hr') hrs = soup.findAll('hr')
for hr in hrs: for hr in hrs:
@ -1019,7 +1079,7 @@ class NYTimes(BasicNewsRecipe):
if bylineauthor: if bylineauthor:
tag = Tag(soup, "h6") tag = Tag(soup, "h6")
tag['class'] = "byline" tag['class'] = "byline"
tag.insert(0, self.fixChars(bylineauthor.renderContents())) tag.insert(0, self.fixChars(self.tag_to_string(bylineauthor,False)))
bylineauthor.replaceWith(tag) bylineauthor.replaceWith(tag)
except: except:
self.log("ERROR: fixing byline author format") self.log("ERROR: fixing byline author format")
@ -1030,7 +1090,7 @@ class NYTimes(BasicNewsRecipe):
if blogcredit: if blogcredit:
tag = Tag(soup, "h6") tag = Tag(soup, "h6")
tag['class'] = "credit" tag['class'] = "credit"
tag.insert(0, self.fixChars(blogcredit.renderContents())) tag.insert(0, self.fixChars(self.tag_to_string(blogcredit,False)))
blogcredit.replaceWith(tag) blogcredit.replaceWith(tag)
except: except:
self.log("ERROR: fixing credit format") self.log("ERROR: fixing credit format")
@ -1084,7 +1144,7 @@ class NYTimes(BasicNewsRecipe):
divTag.replaceWith(tag) divTag.replaceWith(tag)
except: except:
self.log("ERROR: Problem in Add class=authorId to <div> so we can format with CSS") self.log("ERROR: Problem in Add class=authorId to <div> so we can format with CSS")
#print(strftime("%H:%M:%S")+" -- POSTPROCESS TITLE="+self.tag_to_string(soup.title))
return soup return soup
def populate_article_metadata(self, article, soup, first): def populate_article_metadata(self, article, soup, first):

120
recipes/nytimes_sub.recipe
View File

@ -15,6 +15,7 @@ from calibre.ebooks.BeautifulSoup import BeautifulSoup, Tag, BeautifulStoneSoup
class NYTimes(BasicNewsRecipe): class NYTimes(BasicNewsRecipe):
recursions=1 # set this to zero to omit Related articles lists recursions=1 # set this to zero to omit Related articles lists
match_regexps=[r'/[12][0-9][0-9][0-9]/[0-9]+/'] # speeds up processing by preventing index page links from being followed
# set getTechBlogs to True to include the technology blogs # set getTechBlogs to True to include the technology blogs
# set tech_oldest_article to control article age # set tech_oldest_article to control article age
@ -24,6 +25,14 @@ class NYTimes(BasicNewsRecipe):
tech_oldest_article = 14 tech_oldest_article = 14
tech_max_articles_per_feed = 25 tech_max_articles_per_feed = 25
# set getPopularArticles to False if you don't want the Most E-mailed and Most Viewed articles
# otherwise you will get up to 20 of the most popular e-mailed and viewed articles (in each category)
getPopularArticles = True
popularPeriod = '1' # set this to the number of days to include in the measurement
# e.g. 7 will get the most popular measured over the last 7 days
# and 30 will get the most popular measured over 30 days.
# you still only get up to 20 articles in each category
# set headlinesOnly to True for the headlines-only version. If True, webEdition is ignored. # set headlinesOnly to True for the headlines-only version. If True, webEdition is ignored.
headlinesOnly = False headlinesOnly = False
@ -32,7 +41,7 @@ class NYTimes(BasicNewsRecipe):
# number of days old an article can be for inclusion. If oldest_web_article = None all articles # number of days old an article can be for inclusion. If oldest_web_article = None all articles
# will be included. Note: oldest_web_article is ignored if webEdition = False # will be included. Note: oldest_web_article is ignored if webEdition = False
webEdition = False webEdition = False
oldest_web_article = 7 oldest_web_article = None
# download higher resolution images than the small thumbnails typically included in the article # download higher resolution images than the small thumbnails typically included in the article
# the down side of having large beautiful images is the file size is much larger, on the order of 7MB per paper # the down side of having large beautiful images is the file size is much larger, on the order of 7MB per paper
@ -153,7 +162,7 @@ class NYTimes(BasicNewsRecipe):
timefmt = '' timefmt = ''
simultaneous_downloads = 1 #simultaneous_downloads = 1 # no longer required to deal with ads
cover_margins = (18,18,'grey99') cover_margins = (18,18,'grey99')
@ -204,7 +213,8 @@ class NYTimes(BasicNewsRecipe):
re.compile('^subNavigation'), re.compile('^subNavigation'),
re.compile('^leaderboard'), re.compile('^leaderboard'),
re.compile('^module'), re.compile('^module'),
re.compile('commentCount') re.compile('commentCount'),
'credit'
]}), ]}),
dict(name='div', attrs={'class':re.compile('toolsList')}), # bits dict(name='div', attrs={'class':re.compile('toolsList')}), # bits
dict(name='div', attrs={'class':re.compile('postNavigation')}), # bits dict(name='div', attrs={'class':re.compile('postNavigation')}), # bits
@ -291,11 +301,11 @@ class NYTimes(BasicNewsRecipe):
del ans[idx] del ans[idx]
idx_max = idx_max-1 idx_max = idx_max-1
continue continue
if self.verbose: if True: #self.verbose
self.log("Section %s: %d articles" % (ans[idx][0], len(ans[idx][1])) ) self.log("Section %s: %d articles" % (ans[idx][0], len(ans[idx][1])) )
for article in ans[idx][1]: for article in ans[idx][1]:
total_article_count += 1 total_article_count += 1
if self.verbose: if True: #self.verbose
self.log("\t%-40.40s... \t%-60.60s..." % (article['title'].encode('cp1252','replace'), self.log("\t%-40.40s... \t%-60.60s..." % (article['title'].encode('cp1252','replace'),
article['url'].encode('cp1252','replace'))) article['url'].encode('cp1252','replace')))
idx = idx+1 idx = idx+1
@ -351,23 +361,8 @@ class NYTimes(BasicNewsRecipe):
br = BasicNewsRecipe.get_browser() br = BasicNewsRecipe.get_browser()
return br return br
## This doesn't work (and probably never did). It either gets another serve of the advertisement,
## or if it gets the article then get_soup (from which it is invoked) traps trying to do xml decoding.
##
## def skip_ad_pages(self, soup):
## # Skip ad pages served before actual article
## skip_tag = soup.find(True, {'name':'skip'})
## if skip_tag is not None:
## self.log.warn("Found forwarding link: %s" % skip_tag.parent['href'])
## url = 'http://www.nytimes.com' + re.sub(r'\?.*', '', skip_tag.parent['href'])
## url += '?pagewanted=all'
## self.log.warn("Skipping ad to article at '%s'" % url)
## return self.index_to_soup(url, raw=True)
cover_tag = 'NY_NYT' cover_tag = 'NY_NYT'
def get_cover_url(self): def get_cover_url(self):
from datetime import timedelta, date
cover = 'http://webmedia.newseum.org/newseum-multimedia/dfp/jpg'+str(date.today().day)+'/lg/'+self.cover_tag+'.jpg' cover = 'http://webmedia.newseum.org/newseum-multimedia/dfp/jpg'+str(date.today().day)+'/lg/'+self.cover_tag+'.jpg'
br = BasicNewsRecipe.get_browser() br = BasicNewsRecipe.get_browser()
daysback=1 daysback=1
@ -390,6 +385,7 @@ class NYTimes(BasicNewsRecipe):
masthead_url = 'http://graphics8.nytimes.com/images/misc/nytlogo379x64.gif' masthead_url = 'http://graphics8.nytimes.com/images/misc/nytlogo379x64.gif'
def short_title(self): def short_title(self):
return self.title return self.title
@ -398,6 +394,7 @@ class NYTimes(BasicNewsRecipe):
from contextlib import closing from contextlib import closing
import copy import copy
from calibre.ebooks.chardet import xml_to_unicode from calibre.ebooks.chardet import xml_to_unicode
print("ARTICLE_TO_SOUP "+url_or_raw)
if re.match(r'\w+://', url_or_raw): if re.match(r'\w+://', url_or_raw):
br = self.clone_browser(self.browser) br = self.clone_browser(self.browser)
open_func = getattr(br, 'open_novisit', br.open) open_func = getattr(br, 'open_novisit', br.open)
@ -489,6 +486,67 @@ class NYTimes(BasicNewsRecipe):
description=description, author=author, description=description, author=author,
content='')) content=''))
def get_popular_articles(self,ans):
if self.getPopularArticles:
popular_articles = {}
key_list = []
def handleh3(h3tag):
try:
url = h3tag.a['href']
except:
return ('','','','')
url = re.sub(r'\?.*', '', url)
if self.exclude_url(url):
return ('','','','')
url += '?pagewanted=all'
title = self.tag_to_string(h3tag.a,False)
h6tag = h3tag.findNextSibling('h6')
if h6tag is not None:
author = self.tag_to_string(h6tag,False)
else:
author = ''
ptag = h3tag.findNextSibling('p')
if ptag is not None:
desc = self.tag_to_string(ptag,False)
else:
desc = ''
return(title,url,author,desc)
have_emailed = False
emailed_soup = self.index_to_soup('http://www.nytimes.com/most-popular-emailed?period='+self.popularPeriod)
for h3tag in emailed_soup.findAll('h3'):
(title,url,author,desc) = handleh3(h3tag)
if url=='':
continue
if not have_emailed:
key_list.append('Most E-Mailed')
popular_articles['Most E-Mailed'] = []
have_emailed = True
popular_articles['Most E-Mailed'].append(
dict(title=title, url=url, date=strftime('%a, %d %b'),
description=desc, author=author,
content=''))
have_viewed = False
viewed_soup = self.index_to_soup('http://www.nytimes.com/most-popular-viewed?period='+self.popularPeriod)
for h3tag in viewed_soup.findAll('h3'):
(title,url,author,desc) = handleh3(h3tag)
if url=='':
continue
if not have_viewed:
key_list.append('Most Viewed')
popular_articles['Most Viewed'] = []
have_viewed = True
popular_articles['Most Viewed'].append(
dict(title=title, url=url, date=strftime('%a, %d %b'),
description=desc, author=author,
content=''))
viewed_ans = [(k, popular_articles[k]) for k in key_list if popular_articles.has_key(k)]
for x in viewed_ans:
ans.append(x)
return ans
def get_tech_feeds(self,ans): def get_tech_feeds(self,ans):
if self.getTechBlogs: if self.getTechBlogs:
tech_articles = {} tech_articles = {}
@ -550,7 +608,7 @@ class NYTimes(BasicNewsRecipe):
self.handle_article(lidiv) self.handle_article(lidiv)
self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)] self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)]
return self.filter_ans(self.get_tech_feeds(self.ans)) return self.filter_ans(self.get_tech_feeds(self.get_popular_articles(self.ans)))
def parse_todays_index(self): def parse_todays_index(self):
@ -583,7 +641,7 @@ class NYTimes(BasicNewsRecipe):
self.handle_article(lidiv) self.handle_article(lidiv)
self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)] self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)]
return self.filter_ans(self.get_tech_feeds(self.ans)) return self.filter_ans(self.get_tech_feeds(self.get_popular_articles(self.ans)))
def parse_headline_index(self): def parse_headline_index(self):
@ -657,7 +715,7 @@ class NYTimes(BasicNewsRecipe):
self.articles[section_name].append(dict(title=title, url=url, date=pubdate, description=description, author=author, content='')) self.articles[section_name].append(dict(title=title, url=url, date=pubdate, description=description, author=author, content=''))
self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)] self.ans = [(k, self.articles[k]) for k in self.ans if self.articles.has_key(k)]
return self.filter_ans(self.get_tech_feeds(self.ans)) return self.filter_ans(self.get_tech_feeds(self.get_popular_articles(self.ans)))
def parse_index(self): def parse_index(self):
if self.headlinesOnly: if self.headlinesOnly:
@ -745,11 +803,12 @@ class NYTimes(BasicNewsRecipe):
def preprocess_html(self, soup): def preprocess_html(self, soup):
print("PREPROCESS TITLE="+self.tag_to_string(soup.title)) #print(strftime("%H:%M:%S")+" -- PREPROCESS TITLE="+self.tag_to_string(soup.title))
skip_tag = soup.find(True, {'name':'skip'}) skip_tag = soup.find(True, {'name':'skip'})
if skip_tag is not None: if skip_tag is not None:
url = 'http://www.nytimes.com' + re.sub(r'\?.*', '', skip_tag.parent['href']) #url = 'http://www.nytimes.com' + re.sub(r'\?.*', '', skip_tag.parent['href'])
url += '?pagewanted=all' url = 'http://www.nytimes.com' + skip_tag.parent['href']
#url += '?pagewanted=all'
self.log.warn("Skipping ad to article at '%s'" % url) self.log.warn("Skipping ad to article at '%s'" % url)
sleep(5) sleep(5)
soup = self.handle_tags(self.article_to_soup(url)) soup = self.handle_tags(self.article_to_soup(url))
@ -920,6 +979,7 @@ class NYTimes(BasicNewsRecipe):
for aside in soup.findAll('div','aside'): for aside in soup.findAll('div','aside'):
aside.extract() aside.extract()
soup = self.strip_anchors(soup,True) soup = self.strip_anchors(soup,True)
#print("RECURSIVE: "+self.tag_to_string(soup.title))
if soup.find('div',attrs={'id':'blogcontent'}) is None: if soup.find('div',attrs={'id':'blogcontent'}) is None:
if first_fetch: if first_fetch:
@ -1005,7 +1065,7 @@ class NYTimes(BasicNewsRecipe):
if headline: if headline:
tag = Tag(soup, "h2") tag = Tag(soup, "h2")
tag['class'] = "headline" tag['class'] = "headline"
tag.insert(0, self.fixChars(headline.renderContents())) tag.insert(0, self.fixChars(self.tag_to_string(headline,False)))
soup.insert(0, tag) soup.insert(0, tag)
hrs = soup.findAll('hr') hrs = soup.findAll('hr')
for hr in hrs: for hr in hrs:
@ -1019,7 +1079,7 @@ class NYTimes(BasicNewsRecipe):
if bylineauthor: if bylineauthor:
tag = Tag(soup, "h6") tag = Tag(soup, "h6")
tag['class'] = "byline" tag['class'] = "byline"
tag.insert(0, self.fixChars(bylineauthor.renderContents())) tag.insert(0, self.fixChars(self.tag_to_string(bylineauthor,False)))
bylineauthor.replaceWith(tag) bylineauthor.replaceWith(tag)
except: except:
self.log("ERROR: fixing byline author format") self.log("ERROR: fixing byline author format")
@ -1030,7 +1090,7 @@ class NYTimes(BasicNewsRecipe):
if blogcredit: if blogcredit:
tag = Tag(soup, "h6") tag = Tag(soup, "h6")
tag['class'] = "credit" tag['class'] = "credit"
tag.insert(0, self.fixChars(blogcredit.renderContents())) tag.insert(0, self.fixChars(self.tag_to_string(blogcredit,False)))
blogcredit.replaceWith(tag) blogcredit.replaceWith(tag)
except: except:
self.log("ERROR: fixing credit format") self.log("ERROR: fixing credit format")
@ -1084,7 +1144,7 @@ class NYTimes(BasicNewsRecipe):
divTag.replaceWith(tag) divTag.replaceWith(tag)
except: except:
self.log("ERROR: Problem in Add class=authorId to <div> so we can format with CSS") self.log("ERROR: Problem in Add class=authorId to <div> so we can format with CSS")
#print(strftime("%H:%M:%S")+" -- POSTPROCESS TITLE="+self.tag_to_string(soup.title))
return soup return soup
def populate_article_metadata(self, article, soup, first): def populate_article_metadata(self, article, soup, first):

13
src/calibre/customize/builtins.py
View File

@ -1529,6 +1529,15 @@ class StoreNextoStore(StoreBase):
formats = ['EPUB', 'MOBI', 'PDF'] formats = ['EPUB', 'MOBI', 'PDF']
affiliate = True affiliate = True
class StoreNookUKStore(StoreBase):
name = 'Nook UK'
author = 'John Schember'
description = u'Barnes & Noble S.à r.l, a subsidiary of Barnes & Noble, Inc., a leading retailer of content, digital media and educational products, is proud to bring the award-winning NOOK® reading experience and a leading digital bookstore to the UK.'
actual_plugin = 'calibre.gui2.store.stores.nook_uk_plugin:NookUKStore'
headquarters = 'UK'
formats = ['NOOK']
class StoreOpenBooksStore(StoreBase): class StoreOpenBooksStore(StoreBase):
name = 'Open Books' name = 'Open Books'
description = u'Comprehensive listing of DRM free ebooks from a variety of sources provided by users of calibre.' description = u'Comprehensive listing of DRM free ebooks from a variety of sources provided by users of calibre.'
@ -1660,7 +1669,7 @@ plugins += [
StoreAmazonITKindleStore, StoreAmazonITKindleStore,
StoreAmazonUKKindleStore, StoreAmazonUKKindleStore,
StoreBaenWebScriptionStore, StoreBaenWebScriptionStore,
StoreBNStore, StoreSonyStore, StoreBNStore,
StoreBeWriteStore, StoreBeWriteStore,
StoreBiblioStore, StoreBiblioStore,
StoreBookotekaStore, StoreBookotekaStore,
@ -1686,12 +1695,14 @@ plugins += [
StoreMillsBoonUKStore, StoreMillsBoonUKStore,
StoreMobileReadStore, StoreMobileReadStore,
StoreNextoStore, StoreNextoStore,
StoreNookUKStore,
StoreOpenBooksStore, StoreOpenBooksStore,
StoreOzonRUStore, StoreOzonRUStore,
StorePragmaticBookshelfStore, StorePragmaticBookshelfStore,
StorePublioStore, StorePublioStore,
StoreRW2010Store, StoreRW2010Store,
StoreSmashwordsStore, StoreSmashwordsStore,
StoreSonyStore,
StoreVirtualoStore, StoreVirtualoStore,
StoreWaterstonesUKStore, StoreWaterstonesUKStore,
StoreWeightlessBooksStore, StoreWeightlessBooksStore,

6
src/calibre/devices/android/driver.py
View File

@ -48,6 +48,7 @@ class ANDROID(USBMS):
0x2910 : HTC_BCDS, 0x2910 : HTC_BCDS,
0xe77 : HTC_BCDS, 0xe77 : HTC_BCDS,
0xff9 : HTC_BCDS, 0xff9 : HTC_BCDS,
0x0001 : [0x255],
}, },
# Eken # Eken
@ -212,7 +213,8 @@ class ANDROID(USBMS):
'VIZIO', 'GOOGLE', 'FREESCAL', 'KOBO_INC', 'LENOVO', 'ROCKCHIP', 'VIZIO', 'GOOGLE', 'FREESCAL', 'KOBO_INC', 'LENOVO', 'ROCKCHIP',
'POCKET', 'ONDA_MID', 'ZENITHIN', 'INGENIC', 'PMID701C', 'PD', 'POCKET', 'ONDA_MID', 'ZENITHIN', 'INGENIC', 'PMID701C', 'PD',
'PMP5097C', 'MASS', 'NOVO7', 'ZEKI', 'COBY', 'SXZ', 'USB_2.0', 'PMP5097C', 'MASS', 'NOVO7', 'ZEKI', 'COBY', 'SXZ', 'USB_2.0',
'COBY_MID', 'VS', 'AINOL', 'TOPWISE', 'PAD703', 'NEXT8D12'] 'COBY_MID', 'VS', 'AINOL', 'TOPWISE', 'PAD703', 'NEXT8D12',
'MEDIATEK']
WINDOWS_MAIN_MEM = ['ANDROID_PHONE', 'A855', 'A853', 'INC.NEXUS_ONE', WINDOWS_MAIN_MEM = ['ANDROID_PHONE', 'A855', 'A853', 'INC.NEXUS_ONE',
'__UMS_COMPOSITE', '_MB200', 'MASS_STORAGE', '_-_CARD', 'SGH-I897', '__UMS_COMPOSITE', '_MB200', 'MASS_STORAGE', '_-_CARD', 'SGH-I897',
'GT-I9000', 'FILE-STOR_GADGET', 'SGH-T959_CARD', 'SGH-T959', 'SAMSUNG_ANDROID', 'GT-I9000', 'FILE-STOR_GADGET', 'SGH-T959_CARD', 'SGH-T959', 'SAMSUNG_ANDROID',
@ -232,7 +234,7 @@ class ANDROID(USBMS):
'THINKPAD_TABLET', 'SGH-T989', 'YP-G70', 'STORAGE_DEVICE', 'THINKPAD_TABLET', 'SGH-T989', 'YP-G70', 'STORAGE_DEVICE',
'ADVANCED', 'SGH-I727', 'USB_FLASH_DRIVER', 'ANDROID', 'ADVANCED', 'SGH-I727', 'USB_FLASH_DRIVER', 'ANDROID',
'S5830I_CARD', 'MID7042', 'LINK-CREATE', '7035', 'VIEWPAD_7E', 'S5830I_CARD', 'MID7042', 'LINK-CREATE', '7035', 'VIEWPAD_7E',
'NOVO7', 'MB526', '_USB#WYK7MSF8KE', 'TABLET_PC', 'F'] 'NOVO7', 'MB526', '_USB#WYK7MSF8KE', 'TABLET_PC', 'F', 'MT65XX_MS']
WINDOWS_CARD_A_MEM = ['ANDROID_PHONE', 'GT-I9000_CARD', 'SGH-I897', WINDOWS_CARD_A_MEM = ['ANDROID_PHONE', 'GT-I9000_CARD', 'SGH-I897',
'FILE-STOR_GADGET', 'SGH-T959_CARD', 'SGH-T959', 'SAMSUNG_ANDROID', 'GT-P1000_CARD', 'FILE-STOR_GADGET', 'SGH-T959_CARD', 'SGH-T959', 'SAMSUNG_ANDROID', 'GT-P1000_CARD',
'A70S', 'A101IT', '7', 'INCREDIBLE', 'A7EB', 'SGH-T849_CARD', 'A70S', 'A101IT', '7', 'INCREDIBLE', 'A7EB', 'SGH-T849_CARD',

31
src/calibre/ebooks/conversion/plugins/pdf_output.py
View File

@ -91,12 +91,14 @@ class PDFOutput(OutputFormatPlugin):
OptionRecommendation(name='pdf_mono_font_size', OptionRecommendation(name='pdf_mono_font_size',
recommended_value=16, help=_( recommended_value=16, help=_(
'The default font size for monospaced text')), 'The default font size for monospaced text')),
# OptionRecommendation(name='old_pdf_engine', recommended_value=False, OptionRecommendation(name='pdf_mark_links', recommended_value=False,
# help=_('Use the old, less capable engine to generate the PDF')), help=_('Surround all links with a red box, useful for debugging.')),
# OptionRecommendation(name='uncompressed_pdf', OptionRecommendation(name='old_pdf_engine', recommended_value=False,
# recommended_value=False, help=_( help=_('Use the old, less capable engine to generate the PDF')),
# 'Generate an uncompressed PDF, useful for debugging, ' OptionRecommendation(name='uncompressed_pdf',
# 'only works with the new PDF engine.')), recommended_value=False, help=_(
'Generate an uncompressed PDF, useful for debugging, '
'only works with the new PDF engine.')),
]) ])
def convert(self, oeb_book, output_path, input_plugin, opts, log): def convert(self, oeb_book, output_path, input_plugin, opts, log):
@ -190,13 +192,12 @@ class PDFOutput(OutputFormatPlugin):
val[i].value = family_map[k] val[i].value = family_map[k]
def convert_text(self, oeb_book): def convert_text(self, oeb_book):
from calibre.utils.config import tweaks from calibre.ebooks.metadata.opf2 import OPF
if tweaks.get('new_pdf_engine', False): if self.opts.old_pdf_engine:
from calibre.ebooks.pdf.render.from_html import PDFWriter from calibre.ebooks.pdf.writer import PDFWriter
PDFWriter PDFWriter
else: else:
from calibre.ebooks.pdf.writer import PDFWriter from calibre.ebooks.pdf.render.from_html import PDFWriter
from calibre.ebooks.metadata.opf2 import OPF
self.log.debug('Serializing oeb input to disk for processing...') self.log.debug('Serializing oeb input to disk for processing...')
self.get_cover_data() self.get_cover_data()
@ -231,7 +232,15 @@ class PDFOutput(OutputFormatPlugin):
out_stream.seek(0) out_stream.seek(0)
out_stream.truncate() out_stream.truncate()
self.log.debug('Rendering pages to PDF...') self.log.debug('Rendering pages to PDF...')
import time
st = time.time()
if False:
import cProfile
cProfile.runctx('writer.dump(items, out_stream, PDFMetadata(self.metadata))',
globals(), locals(), '/tmp/profile')
else:
writer.dump(items, out_stream, PDFMetadata(self.metadata)) writer.dump(items, out_stream, PDFMetadata(self.metadata))
self.log('Rendered PDF in %g seconds:'%(time.time()-st))
if close: if close:
out_stream.close() out_stream.close()

3
src/calibre/ebooks/html/input.py
View File

@ -17,7 +17,7 @@ from urllib import unquote
from calibre.ebooks.chardet import detect_xml_encoding from calibre.ebooks.chardet import detect_xml_encoding
from calibre.constants import iswindows from calibre.constants import iswindows
from calibre import unicode_path, as_unicode from calibre import unicode_path, as_unicode, replace_entities
class Link(object): class Link(object):
''' '''
@ -147,6 +147,7 @@ class HTMLFile(object):
url = match.group(i) url = match.group(i)
if url: if url:
break break
url = replace_entities(url)
try: try:
link = self.resolve(url) link = self.resolve(url)
except ValueError: except ValueError:

46
src/calibre/ebooks/pdf/render/common.py
View File

@ -9,7 +9,10 @@ __docformat__ = 'restructuredtext en'
import codecs, zlib import codecs, zlib
from io import BytesIO from io import BytesIO
from struct import pack
from calibre.constants import plugins, ispy3
pdf_float = plugins['speedup'][0].pdf_float
EOL = b'\n' EOL = b'\n'
@ -51,15 +54,25 @@ PAPER_SIZES = {k:globals()[k.upper()] for k in ('a0 a1 a2 a3 a4 a5 a6 b0 b1 b2'
# Basic PDF datatypes {{{ # Basic PDF datatypes {{{
ic = str if ispy3 else unicode
icb = (lambda x: str(x).encode('ascii')) if ispy3 else bytes
def fmtnum(o):
if isinstance(o, float):
return pdf_float(o)
return ic(o)
def serialize(o, stream): def serialize(o, stream):
if hasattr(o, 'pdf_serialize'): if isinstance(o, float):
stream.write_raw(pdf_float(o).encode('ascii'))
elif isinstance(o, (int, long)):
stream.write_raw(icb(o))
elif hasattr(o, 'pdf_serialize'):
o.pdf_serialize(stream) o.pdf_serialize(stream)
elif isinstance(o, bool):
stream.write(b'true' if o else b'false')
elif isinstance(o, (int, long, float)):
stream.write(type(u'')(o).encode('ascii'))
elif o is None: elif o is None:
stream.write(b'null') stream.write_raw(b'null')
elif isinstance(o, bool):
stream.write_raw(b'true' if o else b'false')
else: else:
raise ValueError('Unknown object: %r'%o) raise ValueError('Unknown object: %r'%o)
@ -85,19 +98,13 @@ class String(unicode):
raw = codecs.BOM_UTF16_BE + s.encode('utf-16-be') raw = codecs.BOM_UTF16_BE + s.encode('utf-16-be')
stream.write(b'('+raw+b')') stream.write(b'('+raw+b')')
class GlyphIndex(int):
def pdf_serialize(self, stream):
byts = bytearray(pack(b'>H', self))
stream.write('<%s>'%''.join(map(
lambda x: bytes(hex(x)[2:]).rjust(2, b'0'), byts)))
class Dictionary(dict): class Dictionary(dict):
def pdf_serialize(self, stream): def pdf_serialize(self, stream):
stream.write(b'<<' + EOL) stream.write(b'<<' + EOL)
sorted_keys = sorted(self.iterkeys(), sorted_keys = sorted(self.iterkeys(),
key=lambda x:((' ' if x == 'Type' else '')+x)) key=lambda x:({'Type':'1', 'Subtype':'2'}.get(
x, x)+x))
for k in sorted_keys: for k in sorted_keys:
serialize(Name(k), stream) serialize(Name(k), stream)
stream.write(b' ') stream.write(b' ')
@ -161,6 +168,9 @@ class Stream(BytesIO):
super(Stream, self).write(raw if isinstance(raw, bytes) else super(Stream, self).write(raw if isinstance(raw, bytes) else
raw.encode('ascii')) raw.encode('ascii'))
def write_raw(self, raw):
BytesIO.write(self, raw)
class Reference(object): class Reference(object):
def __init__(self, num, obj): def __init__(self, num, obj):
@ -169,5 +179,11 @@ class Reference(object):
def pdf_serialize(self, stream): def pdf_serialize(self, stream):
raw = '%d 0 R'%self.num raw = '%d 0 R'%self.num
stream.write(raw.encode('ascii')) stream.write(raw.encode('ascii'))
def __repr__(self):
return '%d 0 R'%self.num
def __str__(self):
return repr(self)
# }}} # }}}

449
src/calibre/ebooks/pdf/render/engine.py
View File

@ -8,24 +8,27 @@ __copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
__docformat__ = 'restructuredtext en' __docformat__ = 'restructuredtext en'
import sys, traceback import sys, traceback
from math import sqrt
from collections import namedtuple from collections import namedtuple
from functools import wraps, partial from functools import wraps, partial
from future_builtins import map
import sip import sip
from PyQt4.Qt import (QPaintEngine, QPaintDevice, Qt, QApplication, QPainter, from PyQt4.Qt import (QPaintEngine, QPaintDevice, Qt, QTransform, QBrush)
QTransform, QPainterPath, QImage, QByteArray, QBuffer,
qRgba)
from calibre.constants import plugins from calibre.constants import plugins
from calibre.ebooks.pdf.render.serialize import (Color, PDFStream, Path) from calibre.ebooks.pdf.render.serialize import (PDFStream, Path)
from calibre.ebooks.pdf.render.common import inch, A4 from calibre.ebooks.pdf.render.common import inch, A4, fmtnum
from calibre.ebooks.pdf.render.graphics import convert_path, Graphics
from calibre.utils.fonts.sfnt.container import Sfnt from calibre.utils.fonts.sfnt.container import Sfnt
from calibre.utils.fonts.sfnt.metrics import FontMetrics from calibre.utils.fonts.sfnt.metrics import FontMetrics
Point = namedtuple('Point', 'x y') Point = namedtuple('Point', 'x y')
ColorState = namedtuple('ColorState', 'color opacity do') ColorState = namedtuple('ColorState', 'color opacity do')
def repr_transform(t):
vals = map(fmtnum, (t.m11(), t.m12(), t.m21(), t.m22(), t.dx(), t.dy()))
return '[%s]'%' '.join(vals)
def store_error(func): def store_error(func):
@wraps(func) @wraps(func)
@ -38,146 +41,6 @@ def store_error(func):
return errh return errh
class GraphicsState(object): # {{{
def __init__(self):
self.ops = {}
self.initial_state = {
'fill': ColorState(Color(0., 0., 0., 1.), 1.0, False),
'transform': QTransform(),
'dash': [],
'line_width': 0,
'stroke': ColorState(Color(0., 0., 0., 1.), 1.0, True),
'line_cap': 'flat',
'line_join': 'miter',
'clip': (Qt.NoClip, QPainterPath()),
}
self.current_state = self.initial_state.copy()
def reset(self):
self.current_state = self.initial_state.copy()
def update_color_state(self, which, color=None, opacity=None,
brush_style=None, pen_style=None):
current = self.ops.get(which, self.current_state[which])
n = ColorState(*current)
if color is not None:
n = n._replace(color=Color(*color.getRgbF()))
if opacity is not None:
n = n._replace(opacity=opacity)
if opacity is not None:
opacity *= n.color.opacity
if brush_style is not None:
if which == 'fill':
do = (False if opacity == 0.0 or brush_style == Qt.NoBrush else
True)
else:
do = (False if opacity == 0.0 or brush_style == Qt.NoBrush or
pen_style == Qt.NoPen else True)
n = n._replace(do=do)
self.ops[which] = n
def read(self, state):
flags = state.state()
if flags & QPaintEngine.DirtyTransform:
self.ops['transform'] = state.transform()
# TODO: Add support for brush patterns
if flags & QPaintEngine.DirtyBrush:
brush = state.brush()
color = brush.color()
self.update_color_state('fill', color=color,
brush_style=brush.style())
if flags & QPaintEngine.DirtyPen:
pen = state.pen()
brush = pen.brush()
color = pen.color()
self.update_color_state('stroke', color, brush_style=brush.style(),
pen_style=pen.style())
ps = {Qt.DashLine:[3], Qt.DotLine:[1,2], Qt.DashDotLine:[3,2,1,2],
Qt.DashDotDotLine:[3, 2, 1, 2, 1, 2]}.get(pen.style(), [])
self.ops['dash'] = ps
self.ops['line_width'] = pen.widthF()
self.ops['line_cap'] = {Qt.FlatCap:'flat', Qt.RoundCap:'round',
Qt.SquareCap:'square'}.get(pen.capStyle(), 'flat')
self.ops['line_join'] = {Qt.MiterJoin:'miter', Qt.RoundJoin:'round',
Qt.BevelJoin:'bevel'}.get(pen.joinStyle(), 'miter')
if flags & QPaintEngine.DirtyOpacity:
self.update_color_state('fill', opacity=state.opacity())
self.update_color_state('stroke', opacity=state.opacity())
if flags & QPaintEngine.DirtyClipPath or flags & QPaintEngine.DirtyClipRegion:
self.ops['clip'] = True
def __call__(self, engine):
if not self.ops:
return
pdf = engine.pdf
ops = self.ops
current_transform = self.current_state['transform']
transform_changed = 'transform' in ops and ops['transform'] != current_transform
reset_stack = transform_changed or 'clip' in ops
if reset_stack:
pdf.restore_stack()
pdf.save_stack()
# Since we have reset the stack we need to re-apply all previous
# operations, that are different from the default value (clip is
# handled separately).
for op in set(self.initial_state) - {'clip'}:
if op in ops: # These will be applied below
self.current_state[op] = self.initial_state[op]
elif self.current_state[op] != self.initial_state[op]:
self.apply(op, self.current_state[op], engine, pdf)
# Now apply the new operations
for op, val in ops.iteritems():
if op != 'clip' and self.current_state[op] != val:
self.apply(op, val, engine, pdf)
self.current_state[op] = val
if 'clip' in ops:
# Get the current clip
path = engine.painter().clipPath()
if not path.isEmpty():
engine.add_clip(path)
self.ops = {}
def apply(self, op, val, engine, pdf):
getattr(self, 'apply_'+op)(val, engine, pdf)
def apply_transform(self, val, engine, pdf):
if not val.isIdentity():
pdf.transform(val)
def apply_stroke(self, val, engine, pdf):
self.apply_color_state('stroke', val, engine, pdf)
def apply_fill(self, val, engine, pdf):
self.apply_color_state('fill', val, engine, pdf)
def apply_color_state(self, which, val, engine, pdf):
color = val.color._replace(opacity=val.opacity*val.color.opacity)
getattr(pdf, 'set_%s_color'%which)(color)
setattr(engine, 'do_%s'%which, val.do)
def apply_dash(self, val, engine, pdf):
pdf.set_dash(val)
def apply_line_width(self, val, engine, pdf):
pdf.set_line_width(val)
def apply_line_cap(self, val, engine, pdf):
pdf.set_line_cap(val)
def apply_line_join(self, val, engine, pdf):
pdf.set_line_join(val)
# }}}
class Font(FontMetrics): class Font(FontMetrics):
def __init__(self, sfnt): def __init__(self, sfnt):
@ -186,12 +49,21 @@ class Font(FontMetrics):
class PdfEngine(QPaintEngine): class PdfEngine(QPaintEngine):
FEATURES = QPaintEngine.AllFeatures & ~(
QPaintEngine.PorterDuff | QPaintEngine.PerspectiveTransform
| QPaintEngine.ObjectBoundingModeGradients
| QPaintEngine.LinearGradientFill
| QPaintEngine.RadialGradientFill
| QPaintEngine.ConicalGradientFill
)
def __init__(self, file_object, page_width, page_height, left_margin, def __init__(self, file_object, page_width, page_height, left_margin,
top_margin, right_margin, bottom_margin, width, height, top_margin, right_margin, bottom_margin, width, height,
errors=print, debug=print, compress=True): errors=print, debug=print, compress=True,
QPaintEngine.__init__(self, self.features) mark_links=False):
QPaintEngine.__init__(self, self.FEATURES)
self.file_object = file_object self.file_object = file_object
self.compress = compress self.compress, self.mark_links = compress, mark_links
self.page_height, self.page_width = page_height, page_width self.page_height, self.page_width = page_height, page_width
self.left_margin, self.top_margin = left_margin, top_margin self.left_margin, self.top_margin = left_margin, top_margin
self.right_margin, self.bottom_margin = right_margin, bottom_margin self.right_margin, self.bottom_margin = right_margin, bottom_margin
@ -210,49 +82,48 @@ class PdfEngine(QPaintEngine):
self.bottom_margin) / self.pixel_height self.bottom_margin) / self.pixel_height
self.pdf_system = QTransform(sx, 0, 0, -sy, dx, dy) self.pdf_system = QTransform(sx, 0, 0, -sy, dx, dy)
self.do_stroke = True self.graphics = Graphics()
self.do_fill = False
self.scale = sqrt(sy**2 + sx**2)
self.xscale, self.yscale = sx, sy
self.graphics_state = GraphicsState()
self.errors_occurred = False self.errors_occurred = False
self.errors, self.debug = errors, debug self.errors, self.debug = errors, debug
self.fonts = {} self.fonts = {}
i = QImage(1, 1, QImage.Format_ARGB32)
i.fill(qRgba(0, 0, 0, 255))
self.alpha_bit = i.constBits().asstring(4).find(b'\xff')
self.current_page_num = 1 self.current_page_num = 1
self.current_page_inited = False self.current_page_inited = False
self.qt_hack, err = plugins['qt_hack'] self.qt_hack, err = plugins['qt_hack']
if err: if err:
raise RuntimeError('Failed to load qt_hack with err: %s'%err) raise RuntimeError('Failed to load qt_hack with err: %s'%err)
def init_page(self): def apply_graphics_state(self):
self.pdf.transform(self.pdf_system) self.graphics(self.pdf_system, self.painter())
self.pdf.set_rgb_colorspace()
width = self.painter().pen().widthF() if self.isActive() else 0 def resolve_fill(self, rect):
self.pdf.set_line_width(width) self.graphics.resolve_fill(rect, self.pdf_system,
self.do_stroke = True self.painter().transform())
self.do_fill = False
self.graphics_state.reset()
self.pdf.save_stack()
self.current_page_inited = True
@property @property
def features(self): def do_fill(self):
return (QPaintEngine.Antialiasing | QPaintEngine.AlphaBlend | return self.graphics.current_state.do_fill
QPaintEngine.ConstantOpacity | QPaintEngine.PainterPaths |
QPaintEngine.PaintOutsidePaintEvent | @property
QPaintEngine.PrimitiveTransform) def do_stroke(self):
return self.graphics.current_state.do_stroke
def init_page(self):
self.pdf.transform(self.pdf_system)
self.graphics.reset()
self.pdf.save_stack()
self.current_page_inited = True
def begin(self, device): def begin(self, device):
if not hasattr(self, 'pdf'): if not hasattr(self, 'pdf'):
try: try:
self.pdf = PDFStream(self.file_object, (self.page_width, self.pdf = PDFStream(self.file_object, (self.page_width,
self.page_height), self.page_height), compress=self.compress,
compress=self.compress) mark_links=self.mark_links,
debug=self.debug)
self.graphics.begin(self.pdf)
except: except:
self.errors.append(traceback.format_exc()) self.errors(traceback.format_exc())
self.errors_occurred = True
return False return False
return True return True
@ -268,7 +139,8 @@ class PdfEngine(QPaintEngine):
self.end_page() self.end_page()
self.pdf.end() self.pdf.end()
except: except:
self.errors.append(traceback.format_exc()) self.errors(traceback.format_exc())
self.errors_occurred = True
return False return False
finally: finally:
self.pdf = self.file_object = None self.pdf = self.file_object = None
@ -277,139 +149,63 @@ class PdfEngine(QPaintEngine):
def type(self): def type(self):
return QPaintEngine.Pdf return QPaintEngine.Pdf
def add_image(self, img, cache_key):
if img.isNull(): return
return self.pdf.add_image(img, cache_key)
@store_error
def drawTiledPixmap(self, rect, pixmap, point):
self.apply_graphics_state()
brush = QBrush(pixmap)
bl = rect.topLeft()
color, opacity, pattern, do_fill = self.graphics.convert_brush(
brush, bl-point, 1.0, self.pdf_system,
self.painter().transform())
self.pdf.save_stack()
self.pdf.apply_fill(color, pattern)
self.pdf.draw_rect(bl.x(), bl.y(), rect.width(), rect.height(),
stroke=False, fill=True)
self.pdf.restore_stack()
@store_error @store_error
def drawPixmap(self, rect, pixmap, source_rect): def drawPixmap(self, rect, pixmap, source_rect):
self.graphics_state(self) self.apply_graphics_state()
source_rect = source_rect.toRect() source_rect = source_rect.toRect()
pixmap = (pixmap if source_rect == pixmap.rect() else pixmap = (pixmap if source_rect == pixmap.rect() else
pixmap.copy(source_rect)) pixmap.copy(source_rect))
image = pixmap.toImage() image = pixmap.toImage()
ref = self.add_image(image, pixmap.cacheKey()) ref = self.add_image(image, pixmap.cacheKey())
if ref is not None: if ref is not None:
self.pdf.draw_image(rect.x(), rect.height()+rect.y(), rect.width(), self.pdf.draw_image(rect.x(), rect.y(), rect.width(),
-rect.height(), ref) rect.height(), ref)
@store_error @store_error
def drawImage(self, rect, image, source_rect, flags=Qt.AutoColor): def drawImage(self, rect, image, source_rect, flags=Qt.AutoColor):
self.graphics_state(self) self.apply_graphics_state()
source_rect = source_rect.toRect() source_rect = source_rect.toRect()
image = (image if source_rect == image.rect() else image = (image if source_rect == image.rect() else
image.copy(source_rect)) image.copy(source_rect))
ref = self.add_image(image, image.cacheKey()) ref = self.add_image(image, image.cacheKey())
if ref is not None: if ref is not None:
self.pdf.draw_image(rect.x(), rect.height()+rect.y(), rect.width(), self.pdf.draw_image(rect.x(), rect.y(), rect.width(),
-rect.height(), ref) rect.height(), ref)
def add_image(self, img, cache_key):
if img.isNull(): return
ref = self.pdf.get_image(cache_key)
if ref is not None:
return ref
fmt = img.format()
image = QImage(img)
if (image.depth() == 1 and img.colorTable().size() == 2 and
img.colorTable().at(0) == QColor(Qt.black).rgba() and
img.colorTable().at(1) == QColor(Qt.white).rgba()):
if fmt == QImage.Format_MonoLSB:
image = image.convertToFormat(QImage.Format_Mono)
fmt = QImage.Format_Mono
else:
if (fmt != QImage.Format_RGB32 and fmt != QImage.Format_ARGB32):
image = image.convertToFormat(QImage.Format_ARGB32)
fmt = QImage.Format_ARGB32
w = image.width()
h = image.height()
d = image.depth()
if fmt == QImage.Format_Mono:
bytes_per_line = (w + 7) >> 3
data = image.constBits().asstring(bytes_per_line * h)
return self.pdf.write_image(data, w, h, d, cache_key=cache_key)
ba = QByteArray()
buf = QBuffer(ba)
image.save(buf, 'jpeg', 94)
data = bytes(ba.data())
has_alpha = has_mask = False
soft_mask = mask = None
if fmt == QImage.Format_ARGB32:
tmask = image.constBits().asstring(4*w*h)[self.alpha_bit::4]
sdata = bytearray(tmask)
vals = set(sdata)
vals.discard(255)
has_mask = bool(vals)
vals.discard(0)
has_alpha = bool(vals)
if has_alpha:
soft_mask = self.pdf.write_image(tmask, w, h, 8)
elif has_mask:
# dither the soft mask to 1bit and add it. This also helps PDF
# viewers without transparency support
bytes_per_line = (w + 7) >> 3
mdata = bytearray(0 for i in xrange(bytes_per_line * h))
spos = mpos = 0
for y in xrange(h):
for x in xrange(w):
if sdata[spos]:
mdata[mpos + x>>3] |= (0x80 >> (x&7))
spos += 1
mpos += bytes_per_line
mdata = bytes(mdata)
mask = self.pdf.write_image(mdata, w, h, 1)
return self.pdf.write_image(data, w, h, 32, mask=mask, dct=True,
soft_mask=soft_mask, cache_key=cache_key)
@store_error @store_error
def updateState(self, state): def updateState(self, state):
self.graphics_state.read(state) self.graphics.update_state(state, self.painter())
def convert_path(self, path):
p = Path()
i = 0
while i < path.elementCount():
elem = path.elementAt(i)
em = (elem.x, elem.y)
i += 1
if elem.isMoveTo():
p.move_to(*em)
elif elem.isLineTo():
p.line_to(*em)
elif elem.isCurveTo():
added = False
if path.elementCount() > i+1:
c1, c2 = path.elementAt(i), path.elementAt(i+1)
if (c1.type == path.CurveToDataElement and c2.type ==
path.CurveToDataElement):
i += 2
p.curve_to(em[0], em[1], c1.x, c1.y, c2.x, c2.y)
added = True
if not added:
raise ValueError('Invalid curve to operation')
return p
@store_error @store_error
def drawPath(self, path): def drawPath(self, path):
self.graphics_state(self) self.apply_graphics_state()
p = self.convert_path(path) p = convert_path(path)
fill_rule = {Qt.OddEvenFill:'evenodd', fill_rule = {Qt.OddEvenFill:'evenodd',
Qt.WindingFill:'winding'}[path.fillRule()] Qt.WindingFill:'winding'}[path.fillRule()]
self.pdf.draw_path(p, stroke=self.do_stroke, self.pdf.draw_path(p, stroke=self.do_stroke,
fill=self.do_fill, fill_rule=fill_rule) fill=self.do_fill, fill_rule=fill_rule)
def add_clip(self, path):
p = self.convert_path(path)
fill_rule = {Qt.OddEvenFill:'evenodd',
Qt.WindingFill:'winding'}[path.fillRule()]
self.pdf.add_clip(p, fill_rule=fill_rule)
@store_error @store_error
def drawPoints(self, points): def drawPoints(self, points):
self.graphics_state(self) self.apply_graphics_state()
p = Path() p = Path()
for point in points: for point in points:
p.move_to(point.x(), point.y()) p.move_to(point.x(), point.y())
@ -418,8 +214,10 @@ class PdfEngine(QPaintEngine):
@store_error @store_error
def drawRects(self, rects): def drawRects(self, rects):
self.graphics_state(self) self.apply_graphics_state()
with self.graphics:
for rect in rects: for rect in rects:
self.resolve_fill(rect)
bl = rect.topLeft() bl = rect.topLeft()
self.pdf.draw_rect(bl.x(), bl.y(), rect.width(), rect.height(), self.pdf.draw_rect(bl.x(), bl.y(), rect.width(), rect.height(),
stroke=self.do_stroke, fill=self.do_fill) stroke=self.do_stroke, fill=self.do_fill)
@ -438,7 +236,7 @@ class PdfEngine(QPaintEngine):
@store_error @store_error
def drawTextItem(self, point, text_item): def drawTextItem(self, point, text_item):
# super(PdfEngine, self).drawTextItem(point, text_item) # super(PdfEngine, self).drawTextItem(point, text_item)
self.graphics_state(self) self.apply_graphics_state()
gi = self.qt_hack.get_glyphs(point, text_item) gi = self.qt_hack.get_glyphs(point, text_item)
if not gi.indices: if not gi.indices:
sip.delete(gi) sip.delete(gi)
@ -469,7 +267,7 @@ class PdfEngine(QPaintEngine):
@store_error @store_error
def drawPolygon(self, points, mode): def drawPolygon(self, points, mode):
self.graphics_state(self) self.apply_graphics_state()
if not points: return if not points: return
p = Path() p = Path()
p.move_to(points[0].x(), points[0].y()) p.move_to(points[0].x(), points[0].y())
@ -484,20 +282,31 @@ class PdfEngine(QPaintEngine):
def set_metadata(self, *args, **kwargs): def set_metadata(self, *args, **kwargs):
self.pdf.set_metadata(*args, **kwargs) self.pdf.set_metadata(*args, **kwargs)
def __enter__(self): def add_outline(self, toc):
self.pdf.save_stack() self.pdf.links.add_outline(toc)
self.saved_ps = (self.do_stroke, self.do_fill)
def __exit__(self, *args): def add_links(self, current_item, start_page, links, anchors):
self.do_stroke, self.do_fill = self.saved_ps for pos in anchors.itervalues():
self.pdf.restore_stack() pos['left'], pos['top'] = self.pdf_system.map(pos['left'], pos['top'])
for link in links:
pos = link[1]
llx = pos['left']
lly = pos['top'] + pos['height']
urx = pos['left'] + pos['width']
ury = pos['top']
llx, lly = self.pdf_system.map(llx, lly)
urx, ury = self.pdf_system.map(urx, ury)
link[1] = pos['column'] + start_page
link.append((llx, lly, urx, ury))
self.pdf.links.add(current_item, start_page, links, anchors)
class PdfDevice(QPaintDevice): # {{{ class PdfDevice(QPaintDevice): # {{{
def __init__(self, file_object, page_size=A4, left_margin=inch, def __init__(self, file_object, page_size=A4, left_margin=inch,
top_margin=inch, right_margin=inch, bottom_margin=inch, top_margin=inch, right_margin=inch, bottom_margin=inch,
xdpi=1200, ydpi=1200, errors=print, debug=print, compress=True): xdpi=1200, ydpi=1200, errors=print, debug=print,
compress=True, mark_links=False):
QPaintDevice.__init__(self) QPaintDevice.__init__(self)
self.xdpi, self.ydpi = xdpi, ydpi self.xdpi, self.ydpi = xdpi, ydpi
self.page_width, self.page_height = page_size self.page_width, self.page_height = page_size
@ -506,7 +315,10 @@ class PdfDevice(QPaintDevice): # {{{
self.engine = PdfEngine(file_object, self.page_width, self.page_height, self.engine = PdfEngine(file_object, self.page_width, self.page_height,
left_margin, top_margin, right_margin, left_margin, top_margin, right_margin,
bottom_margin, self.width(), self.height(), bottom_margin, self.width(), self.height(),
errors=errors, debug=debug, compress=compress) errors=errors, debug=debug, compress=compress,
mark_links=mark_links)
self.add_outline = self.engine.add_outline
self.add_links = self.engine.add_links
def paintEngine(self): def paintEngine(self):
return self.engine return self.engine
@ -553,59 +365,4 @@ class PdfDevice(QPaintDevice): # {{{
# }}} # }}}
if __name__ == '__main__':
from PyQt4.Qt import (QBrush, QColor, QPoint, QPixmap)
QBrush, QColor, QPoint, QPixmap
app = QApplication([])
p = QPainter()
with open('/tmp/painter.pdf', 'wb') as f:
dev = PdfDevice(f, compress=False)
p.begin(dev)
dev.init_page()
xmax, ymax = p.viewport().width(), p.viewport().height()
try:
p.drawRect(0, 0, xmax, ymax)
# p.drawPolyline(QPoint(0, 0), QPoint(xmax, 0), QPoint(xmax, ymax),
# QPoint(0, ymax), QPoint(0, 0))
# pp = QPainterPath()
# pp.addRect(0, 0, xmax, ymax)
# p.drawPath(pp)
# p.save()
# for i in xrange(3):
# col = [0, 0, 0, 200]
# col[i] = 255
# p.setOpacity(0.3)
# p.setBrush(QBrush(QColor(*col)))
# p.drawRect(0, 0, xmax/10, xmax/10)
# p.translate(xmax/10, xmax/10)
# p.scale(1, 1.5)
# p.restore()
# # p.scale(2, 2)
# # p.rotate(45)
# p.drawPixmap(0, 0, 2048, 2048, QPixmap(I('library.png')))
# p.drawRect(0, 0, 2048, 2048)
# p.save()
# p.drawLine(0, 0, 5000, 0)
# p.rotate(45)
# p.drawLine(0, 0, 5000, 0)
# p.restore()
f = p.font()
f.setPointSize(20)
# f.setLetterSpacing(f.PercentageSpacing, 200)
# f.setUnderline(True)
# f.setOverline(True)
# f.setStrikeOut(True)
f.setFamily('Calibri')
p.setFont(f)
# p.setPen(QColor(0, 0, 255))
# p.scale(2, 2)
# p.rotate(45)
p.drawText(QPoint(300, 300), 'Some—text not By’s ū --- Д AV ff ff')
finally:
p.end()
if dev.engine.errors_occurred:
raise SystemExit(1)

15
src/calibre/ebooks/pdf/render/from_html.py
View File

@ -20,7 +20,6 @@ from calibre.ebooks.oeb.display.webview import load_html
from calibre.ebooks.pdf.render.common import (inch, cm, mm, pica, cicero, from calibre.ebooks.pdf.render.common import (inch, cm, mm, pica, cicero,
didot, PAPER_SIZES) didot, PAPER_SIZES)
from calibre.ebooks.pdf.render.engine import PdfDevice from calibre.ebooks.pdf.render.engine import PdfDevice
from calibre.ebooks.pdf.render.links import Links
def get_page_size(opts, for_comic=False): # {{{ def get_page_size(opts, for_comic=False): # {{{
use_profile = not (opts.override_profile_size or use_profile = not (opts.override_profile_size or
@ -143,7 +142,6 @@ class PDFWriter(QObject):
self.view.page().mainFrame().setScrollBarPolicy(x, self.view.page().mainFrame().setScrollBarPolicy(x,
Qt.ScrollBarAlwaysOff) Qt.ScrollBarAlwaysOff)
self.report_progress = lambda x, y: x self.report_progress = lambda x, y: x
self.links = Links()
def dump(self, items, out_stream, pdf_metadata): def dump(self, items, out_stream, pdf_metadata):
opts = self.opts opts = self.opts
@ -156,7 +154,8 @@ class PDFWriter(QObject):
top_margin=0, right_margin=mr, bottom_margin=0, top_margin=0, right_margin=mr, bottom_margin=0,
xdpi=xdpi, ydpi=ydpi, errors=self.log.error, xdpi=xdpi, ydpi=ydpi, errors=self.log.error,
debug=self.log.debug, compress=not debug=self.log.debug, compress=not
opts.uncompressed_pdf) opts.uncompressed_pdf,
mark_links=opts.pdf_mark_links)
self.page.setViewportSize(QSize(self.doc.width(), self.doc.height())) self.page.setViewportSize(QSize(self.doc.width(), self.doc.height()))
self.render_queue = items self.render_queue = items
@ -187,7 +186,9 @@ class PDFWriter(QObject):
QTimer.singleShot(0, self.render_book) QTimer.singleShot(0, self.render_book)
self.loop.exec_() self.loop.exec_()
# TODO: Outline and links if self.toc is not None and len(self.toc) > 0:
self.doc.add_outline(self.toc)
self.painter.end() self.painter.end()
if self.doc.errors_occurred: if self.doc.errors_occurred:
@ -261,8 +262,7 @@ class PDFWriter(QObject):
amap = self.bridge_value amap = self.bridge_value
if not isinstance(amap, dict): if not isinstance(amap, dict):
amap = {'links':[], 'anchors':{}} # Some javascript error occurred amap = {'links':[], 'anchors':{}} # Some javascript error occurred
self.links.add(self.current_item, self.current_page_num, amap['links'], start_page = self.current_page_num
amap['anchors'])
mf = self.view.page().mainFrame() mf = self.view.page().mainFrame()
while True: while True:
@ -278,3 +278,6 @@ class PDFWriter(QObject):
if self.doc.errors_occurred: if self.doc.errors_occurred:
break break
self.doc.add_links(self.current_item, start_page, amap['links'],
amap['anchors'])

470
src/calibre/ebooks/pdf/render/graphics.py Normal file
View File

@ -0,0 +1,470 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
from math import sqrt
from collections import namedtuple
from PyQt4.Qt import (
QBrush, QPen, Qt, QPointF, QTransform, QPainterPath, QPaintEngine, QImage)
from calibre.ebooks.pdf.render.common import (
Name, Array, fmtnum, Stream, Dictionary)
from calibre.ebooks.pdf.render.serialize import Path
def convert_path(path): # {{{
p = Path()
i = 0
while i < path.elementCount():
elem = path.elementAt(i)
em = (elem.x, elem.y)
i += 1
if elem.isMoveTo():
p.move_to(*em)
elif elem.isLineTo():
p.line_to(*em)
elif elem.isCurveTo():
added = False
if path.elementCount() > i+1:
c1, c2 = path.elementAt(i), path.elementAt(i+1)
if (c1.type == path.CurveToDataElement and c2.type ==
path.CurveToDataElement):
i += 2
p.curve_to(em[0], em[1], c1.x, c1.y, c2.x, c2.y)
added = True
if not added:
raise ValueError('Invalid curve to operation')
return p
# }}}
Brush = namedtuple('Brush', 'origin brush color')
class TilingPattern(Stream):
def __init__(self, cache_key, matrix, w=8, h=8, paint_type=2, compress=False):
Stream.__init__(self, compress=compress)
self.paint_type = paint_type
self.w, self.h = w, h
self.matrix = (matrix.m11(), matrix.m12(), matrix.m21(), matrix.m22(),
matrix.dx(), matrix.dy())
self.resources = Dictionary()
self.cache_key = (self.__class__.__name__, cache_key, self.matrix)
def add_extra_keys(self, d):
d['Type'] = Name('Pattern')
d['PatternType'] = 1
d['PaintType'] = self.paint_type
d['TilingType'] = 1
d['BBox'] = Array([0, 0, self.w, self.h])
d['XStep'] = self.w
d['YStep'] = self.h
d['Matrix'] = Array(self.matrix)
d['Resources'] = self.resources
class QtPattern(TilingPattern):
qt_patterns = ( # {{{
"0 J\n"
"6 w\n"
"[] 0 d\n"
"4 0 m\n"
"4 8 l\n"
"0 4 m\n"
"8 4 l\n"
"S\n", # Dense1Pattern
"0 J\n"
"2 w\n"
"[6 2] 1 d\n"
"0 0 m\n"
"0 8 l\n"
"8 0 m\n"
"8 8 l\n"
"S\n"
"[] 0 d\n"
"2 0 m\n"
"2 8 l\n"
"6 0 m\n"
"6 8 l\n"
"S\n"
"[6 2] -3 d\n"
"4 0 m\n"
"4 8 l\n"
"S\n", # Dense2Pattern
"0 J\n"
"2 w\n"
"[6 2] 1 d\n"
"0 0 m\n"
"0 8 l\n"
"8 0 m\n"
"8 8 l\n"
"S\n"
"[2 2] -1 d\n"
"2 0 m\n"
"2 8 l\n"
"6 0 m\n"
"6 8 l\n"
"S\n"
"[6 2] -3 d\n"
"4 0 m\n"
"4 8 l\n"
"S\n", # Dense3Pattern
"0 J\n"
"2 w\n"
"[2 2] 1 d\n"
"0 0 m\n"
"0 8 l\n"
"8 0 m\n"
"8 8 l\n"
"S\n"
"[2 2] -1 d\n"
"2 0 m\n"
"2 8 l\n"
"6 0 m\n"
"6 8 l\n"
"S\n"
"[2 2] 1 d\n"
"4 0 m\n"
"4 8 l\n"
"S\n", # Dense4Pattern
"0 J\n"
"2 w\n"
"[2 6] -1 d\n"
"0 0 m\n"
"0 8 l\n"
"8 0 m\n"
"8 8 l\n"
"S\n"
"[2 2] 1 d\n"
"2 0 m\n"
"2 8 l\n"
"6 0 m\n"
"6 8 l\n"
"S\n"
"[2 6] 3 d\n"
"4 0 m\n"
"4 8 l\n"
"S\n", # Dense5Pattern
"0 J\n"
"2 w\n"
"[2 6] -1 d\n"
"0 0 m\n"
"0 8 l\n"
"8 0 m\n"
"8 8 l\n"
"S\n"
"[2 6] 3 d\n"
"4 0 m\n"
"4 8 l\n"
"S\n", # Dense6Pattern
"0 J\n"
"2 w\n"
"[2 6] -1 d\n"
"0 0 m\n"
"0 8 l\n"
"8 0 m\n"
"8 8 l\n"
"S\n", # Dense7Pattern
"1 w\n"
"0 4 m\n"
"8 4 l\n"
"S\n", # HorPattern
"1 w\n"
"4 0 m\n"
"4 8 l\n"
"S\n", # VerPattern
"1 w\n"
"4 0 m\n"
"4 8 l\n"
"0 4 m\n"
"8 4 l\n"
"S\n", # CrossPattern
"1 w\n"
"-1 5 m\n"
"5 -1 l\n"
"3 9 m\n"
"9 3 l\n"
"S\n", # BDiagPattern
"1 w\n"
"-1 3 m\n"
"5 9 l\n"
"3 -1 m\n"
"9 5 l\n"
"S\n", # FDiagPattern
"1 w\n"
"-1 3 m\n"
"5 9 l\n"
"3 -1 m\n"
"9 5 l\n"
"-1 5 m\n"
"5 -1 l\n"
"3 9 m\n"
"9 3 l\n"
"S\n", # DiagCrossPattern
) # }}}
def __init__(self, pattern_num, matrix):
super(QtPattern, self).__init__(pattern_num, matrix)
self.write(self.qt_patterns[pattern_num-2])
class TexturePattern(TilingPattern):
def __init__(self, pixmap, matrix, pdf, clone=None):
if clone is None:
image = pixmap.toImage()
cache_key = pixmap.cacheKey()
imgref = pdf.add_image(image, cache_key)
paint_type = (2 if image.format() in {QImage.Format_MonoLSB,
QImage.Format_Mono} else 1)
super(TexturePattern, self).__init__(
cache_key, matrix, w=image.width(), h=image.height(),
paint_type=paint_type)
m = (self.w, 0, 0, -self.h, 0, self.h)
self.resources['XObject'] = Dictionary({'Texture':imgref})
self.write_line('%s cm /Texture Do'%(' '.join(map(fmtnum, m))))
else:
super(TexturePattern, self).__init__(
clone.cache_key[1], matrix, w=clone.w, h=clone.h,
paint_type=clone.paint_type)
self.resources['XObject'] = Dictionary(clone.resources['XObject'])
self.write(clone.getvalue())
class GraphicsState(object):
FIELDS = ('fill', 'stroke', 'opacity', 'transform', 'brush_origin',
'clip', 'do_fill', 'do_stroke')
def __init__(self):
self.fill = QBrush()
self.stroke = QPen()
self.opacity = 1.0
self.transform = QTransform()
self.brush_origin = QPointF()
self.clip = QPainterPath()
self.do_fill = False
self.do_stroke = True
self.qt_pattern_cache = {}
def __eq__(self, other):
for x in self.FIELDS:
if getattr(other, x) != getattr(self, x):
return False
return True
def copy(self):
ans = GraphicsState()
ans.fill = QBrush(self.fill)
ans.stroke = QPen(self.stroke)
ans.opacity = self.opacity
ans.transform = self.transform * QTransform()
ans.brush_origin = QPointF(self.brush_origin)
ans.clip = self.clip
ans.do_fill, ans.do_stroke = self.do_fill, self.do_stroke
return ans
class Graphics(object):
def __init__(self):
self.base_state = GraphicsState()
self.current_state = GraphicsState()
self.pending_state = None
def begin(self, pdf):
self.pdf = pdf
def update_state(self, state, painter):
flags = state.state()
if self.pending_state is None:
self.pending_state = self.current_state.copy()
s = self.pending_state
if flags & QPaintEngine.DirtyTransform:
s.transform = state.transform()
if flags & QPaintEngine.DirtyBrushOrigin:
s.brush_origin = state.brushOrigin()
if flags & QPaintEngine.DirtyBrush:
s.fill = state.brush()
if flags & QPaintEngine.DirtyPen:
s.stroke = state.pen()
if flags & QPaintEngine.DirtyOpacity:
s.opacity = state.opacity()
if flags & QPaintEngine.DirtyClipPath or flags & QPaintEngine.DirtyClipRegion:
s.clip = painter.clipPath()
def reset(self):
self.current_state = GraphicsState()
self.pending_state = None
def __call__(self, pdf_system, painter):
# Apply the currently pending state to the PDF
if self.pending_state is None:
return
pdf_state = self.current_state
ps = self.pending_state
pdf = self.pdf
if (ps.transform != pdf_state.transform or ps.clip != pdf_state.clip):
pdf.restore_stack()
pdf.save_stack()
pdf_state = self.base_state
if (pdf_state.transform != ps.transform):
pdf.transform(ps.transform)
if (pdf_state.opacity != ps.opacity or pdf_state.stroke != ps.stroke):
self.apply_stroke(ps, pdf_system, painter)
if (pdf_state.opacity != ps.opacity or pdf_state.fill != ps.fill or
pdf_state.brush_origin != ps.brush_origin):
self.apply_fill(ps, pdf_system, painter)
if (pdf_state.clip != ps.clip):
p = convert_path(ps.clip)
fill_rule = {Qt.OddEvenFill:'evenodd',
Qt.WindingFill:'winding'}[ps.clip.fillRule()]
pdf.add_clip(p, fill_rule=fill_rule)
self.current_state = self.pending_state
self.pending_state = None
def convert_brush(self, brush, brush_origin, global_opacity,
pdf_system, qt_system):
# Convert a QBrush to PDF operators
style = brush.style()
pdf = self.pdf
pattern = color = pat = None
opacity = 1.0
do_fill = True
matrix = (QTransform.fromTranslate(brush_origin.x(), brush_origin.y())
* pdf_system * qt_system.inverted()[0])
vals = list(brush.color().getRgbF())
self.brushobj = None
if style <= Qt.DiagCrossPattern:
opacity = global_opacity * vals[-1]
color = vals[:3]
if style > Qt.SolidPattern:
pat = QtPattern(style, matrix)
pattern = pdf.add_pattern(pat)
if opacity < 1e-4 or style == Qt.NoBrush:
do_fill = False
elif style == Qt.TexturePattern:
pat = TexturePattern(brush.texture(), matrix, pdf)
opacity = global_opacity
if pat.paint_type == 2:
opacity *= vals[-1]
color = vals[:3]
pattern = pdf.add_pattern(pat)
if opacity < 1e-4 or style == Qt.NoBrush:
do_fill = False
self.brushobj = Brush(brush_origin, pat, color)
# TODO: Add support for gradient fills
return color, opacity, pattern, do_fill
def apply_stroke(self, state, pdf_system, painter):
# TODO: Support miter limit by using QPainterPathStroker
pen = state.stroke
self.pending_state.do_stroke = True
pdf = self.pdf
# Width
w = pen.widthF()
if pen.isCosmetic():
t = painter.transform()
w /= sqrt(t.m11()**2 + t.m22()**2)
pdf.serialize(w)
pdf.current_page.write(' w ')
# Line cap
cap = {Qt.FlatCap:0, Qt.RoundCap:1, Qt.SquareCap:
2}.get(pen.capStyle(), 0)
pdf.current_page.write('%d J '%cap)
# Line join
join = {Qt.MiterJoin:0, Qt.RoundJoin:1,
Qt.BevelJoin:2}.get(pen.joinStyle(), 0)
pdf.current_page.write('%d j '%join)
# Dash pattern
ps = {Qt.DashLine:[3], Qt.DotLine:[1,2], Qt.DashDotLine:[3,2,1,2],
Qt.DashDotDotLine:[3, 2, 1, 2, 1, 2]}.get(pen.style(), [])
if ps:
pdf.serialize(Array(ps))
pdf.current_page.write(' 0 d ')
# Stroke fill
color, opacity, pattern, self.pending_state.do_stroke = self.convert_brush(
pen.brush(), state.brush_origin, state.opacity, pdf_system,
painter.transform())
self.pdf.apply_stroke(color, pattern, opacity)
if pen.style() == Qt.NoPen:
self.pending_state.do_stroke = False
def apply_fill(self, state, pdf_system, painter):
self.pending_state.do_fill = True
color, opacity, pattern, self.pending_state.do_fill = self.convert_brush(
state.fill, state.brush_origin, state.opacity, pdf_system,
painter.transform())
self.pdf.apply_fill(color, pattern, opacity)
self.last_fill = self.brushobj
def __enter__(self):
self.pdf.save_stack()
def __exit__(self, *args):
self.pdf.restore_stack()
def resolve_fill(self, rect, pdf_system, qt_system):
'''
Qt's paint system does not update brushOrigin when using
TexturePatterns and it also uses TexturePatterns to emulate gradients,
leading to brokenness. So this method allows the paint engine to update
the brush origin before painting an object. While not perfect, this is
better than nothing.
'''
if not hasattr(self, 'last_fill') or not self.current_state.do_fill:
return
if isinstance(self.last_fill.brush, TexturePattern):
tl = rect.topLeft()
if tl == self.last_fill.origin:
return
matrix = (QTransform.fromTranslate(tl.x(), tl.y())
* pdf_system * qt_system.inverted()[0])
pat = TexturePattern(None, matrix, self.pdf, clone=self.last_fill.brush)
pattern = self.pdf.add_pattern(pat)
self.pdf.apply_fill(self.last_fill.color, pattern)

102
src/calibre/ebooks/pdf/render/links.py
View File

@ -8,25 +8,115 @@ __copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
__docformat__ = 'restructuredtext en' __docformat__ = 'restructuredtext en'
import os import os
from future_builtins import map
from urlparse import urlparse, urlunparse
from urllib2 import quote, unquote
from calibre.ebooks.pdf.render.common import Array, Name from calibre.ebooks.pdf.render.common import Array, Name, Dictionary, String
class Destination(Array): class Destination(Array):
def __init__(self, start_page, pos): def __init__(self, start_page, pos, get_pageref):
super(Destination, self).__init__( super(Destination, self).__init__(
[start_page + pos['column'], Name('FitH'), pos['y']]) [get_pageref(start_page + pos['column']), Name('XYZ'), pos['left'],
pos['top'], None]
)
class Links(object): class Links(object):
def __init__(self): def __init__(self, pdf, mark_links, page_size):
self.anchors = {} self.anchors = {}
self.links = []
self.start = {'top':page_size[1], 'column':0, 'left':0}
self.pdf = pdf
self.mark_links = mark_links
def add(self, base_path, start_page, links, anchors): def add(self, base_path, start_page, links, anchors):
path = os.path.normcase(os.path.abspath(base_path)) path = os.path.normcase(os.path.abspath(base_path))
self.anchors[path] = a = {} self.anchors[path] = a = {}
a[None] = Destination(start_page, {'y':0, 'column':0}) a[None] = Destination(start_page, self.start, self.pdf.get_pageref)
for anchor, pos in anchors.iteritems(): for anchor, pos in anchors.iteritems():
a[anchor] = Destination(start_page, pos) a[anchor] = Destination(start_page, pos, self.pdf.get_pageref)
for link in links:
href, page, rect = link
p, frag = href.partition('#')[0::2]
link = ((path, p, frag or None), self.pdf.get_pageref(page).obj, Array(rect))
self.links.append(link)
def add_links(self):
for link in self.links:
path, href, frag = link[0]
page, rect = link[1:]
combined_path = os.path.abspath(os.path.join(os.path.dirname(path), *href.split('/')))
is_local = not href or combined_path in self.anchors
annot = Dictionary({
'Type':Name('Annot'), 'Subtype':Name('Link'),
'Rect':rect, 'Border':Array([0,0,0]),
})
if self.mark_links:
annot.update({'Border':Array([16, 16, 1]), 'C':Array([1.0, 0,
0])})
if is_local:
path = combined_path if href else path
annot['Dest'] = self.anchors[path][frag]
else:
url = href + (('#'+frag) if frag else '')
purl = urlparse(url)
if purl.scheme and purl.scheme != 'file':
action = Dictionary({
'Type':Name('Action'), 'S':Name('URI'),
})
parts = (x.encode('utf-8') if isinstance(x, type(u'')) else
x for x in purl)
url = urlunparse(map(quote, map(unquote,
parts))).decode('ascii')
action['URI'] = String(url)
annot['A'] = action
if 'A' in annot or 'Dest' in annot:
if 'Annots' not in page:
page['Annots'] = Array()
page['Annots'].append(self.pdf.objects.add(annot))
else:
self.pdf.debug('Could not find destination for link: %s in file %s'%
(href, path))
def add_outline(self, toc):
parent = Dictionary({'Type':Name('Outlines')})
parentref = self.pdf.objects.add(parent)
self.process_children(toc, parentref, parent_is_root=True)
self.pdf.catalog.obj['Outlines'] = parentref
def process_children(self, toc, parentref, parent_is_root=False):
childrefs = []
for child in toc:
childref = self.process_toc_item(child, parentref)
if childref is None:
continue
if childrefs:
childrefs[-1].obj['Next'] = childref
childref.obj['Prev'] = childrefs[-1]
childrefs.append(childref)
if len(child) > 0:
self.process_children(child, childref)
if childrefs:
parentref.obj['First'] = childrefs[0]
parentref.obj['Last'] = childrefs[-1]
if not parent_is_root:
parentref.obj['Count'] = -len(childrefs)
def process_toc_item(self, toc, parentref):
path = toc.abspath or None
frag = toc.fragment or None
if path is None:
return
path = os.path.normcase(os.path.abspath(path))
if path not in self.anchors:
return None
a = self.anchors[path]
dest = a.get(frag, a[None])
item = Dictionary({'Parent':parentref, 'Dest':dest,
'Title':String(toc.text or _('Unknown'))})
return self.pdf.objects.add(item)

242
src/calibre/ebooks/pdf/render/serialize.py
View File

@ -9,19 +9,18 @@ __docformat__ = 'restructuredtext en'
import hashlib import hashlib
from future_builtins import map from future_builtins import map
from itertools import izip
from collections import namedtuple from PyQt4.Qt import QBuffer, QByteArray, QImage, Qt, QColor, qRgba
from calibre.constants import (__appname__, __version__) from calibre.constants import (__appname__, __version__)
from calibre.ebooks.pdf.render.common import ( from calibre.ebooks.pdf.render.common import (
Reference, EOL, serialize, Stream, Dictionary, String, Name, Array, Reference, EOL, serialize, Stream, Dictionary, String, Name, Array,
GlyphIndex) fmtnum)
from calibre.ebooks.pdf.render.fonts import FontManager from calibre.ebooks.pdf.render.fonts import FontManager
from calibre.ebooks.pdf.render.links import Links
PDFVER = b'%PDF-1.3' PDFVER = b'%PDF-1.3'
Color = namedtuple('Color', 'red green blue opacity')
class IndirectObjects(object): class IndirectObjects(object):
def __init__(self): def __init__(self):
@ -89,6 +88,7 @@ class Page(Stream):
self.opacities = {} self.opacities = {}
self.fonts = {} self.fonts = {}
self.xobjects = {} self.xobjects = {}
self.patterns = {}
def set_opacity(self, opref): def set_opacity(self, opref):
if opref not in self.opacities: if opref not in self.opacities:
@ -107,6 +107,11 @@ class Page(Stream):
self.xobjects[imgref] = 'Image%d'%len(self.xobjects) self.xobjects[imgref] = 'Image%d'%len(self.xobjects)
return self.xobjects[imgref] return self.xobjects[imgref]
def add_pattern(self, patternref):
if patternref not in self.patterns:
self.patterns[patternref] = 'Pat%d'%len(self.patterns)
return self.patterns[patternref]
def add_resources(self): def add_resources(self):
r = Dictionary() r = Dictionary()
if self.opacities: if self.opacities:
@ -124,6 +129,13 @@ class Page(Stream):
for ref, name in self.xobjects.iteritems(): for ref, name in self.xobjects.iteritems():
xobjects[name] = ref xobjects[name] = ref
r['XObject'] = xobjects r['XObject'] = xobjects
if self.patterns:
r['ColorSpace'] = Dictionary({'PCSp':Array(
[Name('Pattern'), Name('DeviceRGB')])})
patterns = Dictionary()
for ref, name in self.patterns.iteritems():
patterns[name] = ref
r['Pattern'] = patterns
if r: if r:
self.page_dict['Resources'] = r self.page_dict['Resources'] = r
@ -153,54 +165,6 @@ class Path(object):
def close(self): def close(self):
self.ops.append(('h',)) self.ops.append(('h',))
class Text(object):
def __init__(self):
self.transform = self.default_transform = [1, 0, 0, 1, 0, 0]
self.font_name = 'Times-Roman'
self.font_path = None
self.horizontal_scale = self.default_horizontal_scale = 100
self.word_spacing = self.default_word_spacing = 0
self.char_space = self.default_char_space = 0
self.glyph_adjust = self.default_glyph_adjust = None
self.size = 12
self.text = ''
def set_transform(self, *args):
if len(args) == 1:
m = args[0]
vals = [m.m11(), m.m12(), m.m21(), m.m22(), m.dx(), m.dy()]
else:
vals = args
self.transform = vals
def pdf_serialize(self, stream, font_name):
if not self.text: return
stream.write_line('BT ')
serialize(Name(font_name), stream)
stream.write(' %g Tf '%self.size)
stream.write(' '.join(map(type(u''), self.transform)) + ' Tm ')
if self.horizontal_scale != self.default_horizontal_scale:
stream.write('%g Tz '%self.horizontal_scale)
if self.word_spacing != self.default_word_spacing:
stream.write('%g Tw '%self.word_spacing)
if self.char_space != self.default_char_space:
stream.write('%g Tc '%self.char_space)
stream.write_line()
if self.glyph_adjust is self.default_glyph_adjust:
serialize(String(self.text), stream)
stream.write(' Tj ')
else:
chars = Array()
frac, widths = self.glyph_adjust
for c, width in izip(self.text, widths):
chars.append(String(c))
chars.append(int(width * frac))
serialize(chars, stream)
stream.write(' TJ ')
stream.write_line('ET')
class Catalog(Dictionary): class Catalog(Dictionary):
def __init__(self, pagetree): def __init__(self, pagetree):
@ -219,6 +183,9 @@ class PageTree(Dictionary):
self['Kids'].append(pageref) self['Kids'].append(pageref)
self['Count'] += 1 self['Count'] += 1
def get_ref(self, num):
return self['Kids'][num-1]
class HashingStream(object): class HashingStream(object):
def __init__(self, f): def __init__(self, f):
@ -228,7 +195,9 @@ class HashingStream(object):
self.last_char = b'' self.last_char = b''
def write(self, raw): def write(self, raw):
raw = raw if isinstance(raw, bytes) else raw.encode('ascii') self.write_raw(raw if isinstance(raw, bytes) else raw.encode('ascii'))
def write_raw(self, raw):
self.f.write(raw) self.f.write(raw)
self.hashobj.update(raw) self.hashobj.update(raw)
if raw: if raw:
@ -277,7 +246,8 @@ class PDFStream(object):
( True, True, 'evenodd') : 'B*', ( True, True, 'evenodd') : 'B*',
} }
def __init__(self, stream, page_size, compress=False): def __init__(self, stream, page_size, compress=False, mark_links=False,
debug=print):
self.stream = HashingStream(stream) self.stream = HashingStream(stream)
self.compress = compress self.compress = compress
self.write_line(PDFVER) self.write_line(PDFVER)
@ -294,6 +264,12 @@ class PDFStream(object):
self.stroke_opacities, self.fill_opacities = {}, {} self.stroke_opacities, self.fill_opacities = {}, {}
self.font_manager = FontManager(self.objects, self.compress) self.font_manager = FontManager(self.objects, self.compress)
self.image_cache = {} self.image_cache = {}
self.pattern_cache = {}
self.debug = debug
self.links = Links(self, mark_links, page_size)
i = QImage(1, 1, QImage.Format_ARGB32)
i.fill(qRgba(0, 0, 0, 255))
self.alpha_bit = i.constBits().asstring(4).find(b'\xff')
@property @property
def page_tree(self): def page_tree(self):
@ -303,6 +279,9 @@ class PDFStream(object):
def catalog(self): def catalog(self):
return self.objects[1] return self.objects[1]
def get_pageref(self, pagenum):
return self.page_tree.obj.get_ref(pagenum)
def set_metadata(self, title=None, author=None, tags=None): def set_metadata(self, title=None, author=None, tags=None):
if title: if title:
self.info['Title'] = String(title) self.info['Title'] = String(title)
@ -321,12 +300,9 @@ class PDFStream(object):
vals = [m.m11(), m.m12(), m.m21(), m.m22(), m.dx(), m.dy()] vals = [m.m11(), m.m12(), m.m21(), m.m22(), m.dx(), m.dy()]
else: else:
vals = args vals = args
cm = ' '.join(map(type(u''), vals)) cm = ' '.join(map(fmtnum, vals))
self.current_page.write_line(cm + ' cm') self.current_page.write_line(cm + ' cm')
def set_rgb_colorspace(self):
self.current_page.write_line('/DeviceRGB CS /DeviceRGB cs')
def save_stack(self): def save_stack(self):
self.current_page.write_line('q') self.current_page.write_line('q')
@ -337,7 +313,7 @@ class PDFStream(object):
self.current_page.write_line('Q q') self.current_page.write_line('Q q')
def draw_rect(self, x, y, width, height, stroke=True, fill=False): def draw_rect(self, x, y, width, height, stroke=True, fill=False):
self.current_page.write('%g %g %g %g re '%(x, y, width, height)) self.current_page.write('%s re '%' '.join(map(fmtnum, (x, y, width, height))))
self.current_page.write_line(self.PATH_OPS[(stroke, fill, 'winding')]) self.current_page.write_line(self.PATH_OPS[(stroke, fill, 'winding')])
def write_path(self, path): def write_path(self, path):
@ -345,7 +321,8 @@ class PDFStream(object):
if i != 0: if i != 0:
self.current_page.write_line() self.current_page.write_line()
for x in op: for x in op:
self.current_page.write(type(u'')(x) + ' ') self.current_page.write(
(fmtnum(x) if isinstance(x, (int, long, float)) else x) + ' ')
def draw_path(self, path, stroke=True, fill=False, fill_rule='winding'): def draw_path(self, path, stroke=True, fill=False, fill_rule='winding'):
if not path.ops: return if not path.ops: return
@ -358,67 +335,38 @@ class PDFStream(object):
op = 'W' if fill_rule == 'winding' else 'W*' op = 'W' if fill_rule == 'winding' else 'W*'
self.current_page.write_line(op + ' ' + 'n') self.current_page.write_line(op + ' ' + 'n')
def set_dash(self, array, phase=0): def serialize(self, o):
array = Array(array) serialize(o, self.current_page)
serialize(array, self.current_page)
self.current_page.write(b' ')
serialize(phase, self.current_page)
self.current_page.write_line(' d')
def set_line_width(self, width): def set_stroke_opacity(self, opacity):
serialize(width, self.current_page)
self.current_page.write_line(' w')
def set_line_cap(self, style):
serialize({'flat':0, 'round':1, 'square':2}.get(style),
self.current_page)
self.current_page.write_line(' J')
def set_line_join(self, style):
serialize({'miter':0, 'round':1, 'bevel':2}[style], self.current_page)
self.current_page.write_line(' j')
def set_stroke_color(self, color):
opacity = color.opacity
if opacity not in self.stroke_opacities: if opacity not in self.stroke_opacities:
op = Dictionary({'Type':Name('ExtGState'), 'CA': opacity}) op = Dictionary({'Type':Name('ExtGState'), 'CA': opacity})
self.stroke_opacities[opacity] = self.objects.add(op) self.stroke_opacities[opacity] = self.objects.add(op)
self.current_page.set_opacity(self.stroke_opacities[opacity]) self.current_page.set_opacity(self.stroke_opacities[opacity])
self.current_page.write_line(' '.join(map(type(u''), color[:3])) + ' SC')
def set_fill_color(self, color): def set_fill_opacity(self, opacity):
opacity = color.opacity opacity = float(opacity)
if opacity not in self.fill_opacities: if opacity not in self.fill_opacities:
op = Dictionary({'Type':Name('ExtGState'), 'ca': opacity}) op = Dictionary({'Type':Name('ExtGState'), 'ca': opacity})
self.fill_opacities[opacity] = self.objects.add(op) self.fill_opacities[opacity] = self.objects.add(op)
self.current_page.set_opacity(self.fill_opacities[opacity]) self.current_page.set_opacity(self.fill_opacities[opacity])
self.current_page.write_line(' '.join(map(type(u''), color[:3])) + ' sc')
def end_page(self): def end_page(self):
pageref = self.current_page.end(self.objects, self.stream) pageref = self.current_page.end(self.objects, self.stream)
self.page_tree.obj.add_page(pageref) self.page_tree.obj.add_page(pageref)
self.current_page = Page(self.page_tree, compress=self.compress) self.current_page = Page(self.page_tree, compress=self.compress)
def draw_text(self, text_object):
if text_object.font_path is None:
fontref = self.font_manager.add_standard_font(text_object.font_name)
else:
raise NotImplementedError()
name = self.current_page.add_font(fontref)
text_object.pdf_serialize(self.current_page, name)
def draw_glyph_run(self, transform, size, font_metrics, glyphs): def draw_glyph_run(self, transform, size, font_metrics, glyphs):
glyph_ids = {x[-1] for x in glyphs} glyph_ids = {x[-1] for x in glyphs}
fontref = self.font_manager.add_font(font_metrics, glyph_ids) fontref = self.font_manager.add_font(font_metrics, glyph_ids)
name = self.current_page.add_font(fontref) name = self.current_page.add_font(fontref)
self.current_page.write(b'BT ') self.current_page.write(b'BT ')
serialize(Name(name), self.current_page) serialize(Name(name), self.current_page)
self.current_page.write(' %g Tf '%size) self.current_page.write(' %s Tf '%fmtnum(size))
self.current_page.write('%s Tm '%' '.join(map(type(u''), transform))) self.current_page.write('%s Tm '%' '.join(map(fmtnum, transform)))
for x, y, glyph_id in glyphs: for x, y, glyph_id in glyphs:
self.current_page.write('%g %g Td '%(x, y)) self.current_page.write_raw(('%s %s Td <%04X> Tj '%(
serialize(GlyphIndex(glyph_id), self.current_page) fmtnum(x), fmtnum(y), glyph_id)).encode('ascii'))
self.current_page.write(' Tj ')
self.current_page.write_line(b' ET') self.current_page.write_line(b' ET')
def get_image(self, cache_key): def get_image(self, cache_key):
@ -431,17 +379,109 @@ class PDFStream(object):
self.objects.commit(r, self.stream) self.objects.commit(r, self.stream)
return r return r
def draw_image(self, x, y, xscale, yscale, imgref): def add_image(self, img, cache_key):
ref = self.get_image(cache_key)
if ref is not None:
return ref
fmt = img.format()
image = QImage(img)
if (image.depth() == 1 and img.colorTable().size() == 2 and
img.colorTable().at(0) == QColor(Qt.black).rgba() and
img.colorTable().at(1) == QColor(Qt.white).rgba()):
if fmt == QImage.Format_MonoLSB:
image = image.convertToFormat(QImage.Format_Mono)
fmt = QImage.Format_Mono
else:
if (fmt != QImage.Format_RGB32 and fmt != QImage.Format_ARGB32):
image = image.convertToFormat(QImage.Format_ARGB32)
fmt = QImage.Format_ARGB32
w = image.width()
h = image.height()
d = image.depth()
if fmt == QImage.Format_Mono:
bytes_per_line = (w + 7) >> 3
data = image.constBits().asstring(bytes_per_line * h)
return self.write_image(data, w, h, d, cache_key=cache_key)
ba = QByteArray()
buf = QBuffer(ba)
image.save(buf, 'jpeg', 94)
data = bytes(ba.data())
has_alpha = has_mask = False
soft_mask = mask = None
if fmt == QImage.Format_ARGB32:
tmask = image.constBits().asstring(4*w*h)[self.alpha_bit::4]
sdata = bytearray(tmask)
vals = set(sdata)
vals.discard(255)
has_mask = bool(vals)
vals.discard(0)
has_alpha = bool(vals)
if has_alpha:
soft_mask = self.write_image(tmask, w, h, 8)
elif has_mask:
# dither the soft mask to 1bit and add it. This also helps PDF
# viewers without transparency support
bytes_per_line = (w + 7) >> 3
mdata = bytearray(0 for i in xrange(bytes_per_line * h))
spos = mpos = 0
for y in xrange(h):
for x in xrange(w):
if sdata[spos]:
mdata[mpos + x>>3] |= (0x80 >> (x&7))
spos += 1
mpos += bytes_per_line
mdata = bytes(mdata)
mask = self.write_image(mdata, w, h, 1)
return self.write_image(data, w, h, 32, mask=mask, dct=True,
soft_mask=soft_mask, cache_key=cache_key)
def add_pattern(self, pattern):
if pattern.cache_key not in self.pattern_cache:
self.pattern_cache[pattern.cache_key] = self.objects.add(pattern)
return self.current_page.add_pattern(self.pattern_cache[pattern.cache_key])
def draw_image(self, x, y, width, height, imgref):
name = self.current_page.add_image(imgref) name = self.current_page.add_image(imgref)
self.current_page.write('q %g 0 0 %g %g %g cm '%(xscale, yscale, x, y)) self.current_page.write('q %s 0 0 %s %s %s cm '%(fmtnum(width),
fmtnum(-height), fmtnum(x), fmtnum(y+height)))
serialize(Name(name), self.current_page) serialize(Name(name), self.current_page)
self.current_page.write_line(' Do Q') self.current_page.write_line(' Do Q')
def apply_color_space(self, color, pattern, stroke=False):
wl = self.current_page.write_line
if color is not None and pattern is None:
wl(' '.join(map(fmtnum, color)) + (' RG' if stroke else ' rg'))
elif color is None and pattern is not None:
wl('/Pattern %s /%s %s'%('CS' if stroke else 'cs', pattern,
'SCN' if stroke else 'scn'))
elif color is not None and pattern is not None:
col = ' '.join(map(fmtnum, color))
wl('/PCSp %s %s /%s %s'%('CS' if stroke else 'cs', col, pattern,
'SCN' if stroke else 'scn'))
def apply_fill(self, color=None, pattern=None, opacity=None):
if opacity is not None:
self.set_fill_opacity(opacity)
self.apply_color_space(color, pattern)
def apply_stroke(self, color=None, pattern=None, opacity=None):
if opacity is not None:
self.set_stroke_opacity(opacity)
self.apply_color_space(color, pattern, stroke=True)
def end(self): def end(self):
if self.current_page.getvalue(): if self.current_page.getvalue():
self.end_page() self.end_page()
self.font_manager.embed_fonts() self.font_manager.embed_fonts()
inforef = self.objects.add(self.info) inforef = self.objects.add(self.info)
self.links.add_links()
self.objects.pdf_serialize(self.stream) self.objects.pdf_serialize(self.stream)
self.write_line() self.write_line()
startxref = self.objects.write_xref(self.stream) startxref = self.objects.write_xref(self.stream)

128
src/calibre/ebooks/pdf/render/test.py Normal file
View File

@ -0,0 +1,128 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import os
from tempfile import gettempdir
from PyQt4.Qt import (QBrush, QColor, QPoint, QPixmap, QPainterPath, QRectF,
QApplication, QPainter, Qt, QImage, QLinearGradient,
QPointF, QPen)
QBrush, QColor, QPoint, QPixmap, QPainterPath, QRectF, Qt, QPointF
from calibre.ebooks.pdf.render.engine import PdfDevice
def full(p, xmax, ymax):
p.drawRect(0, 0, xmax, ymax)
p.drawPolyline(QPoint(0, 0), QPoint(xmax, 0), QPoint(xmax, ymax),
QPoint(0, ymax), QPoint(0, 0))
pp = QPainterPath()
pp.addRect(0, 0, xmax, ymax)
p.drawPath(pp)
p.save()
for i in xrange(3):
col = [0, 0, 0, 200]
col[i] = 255
p.setOpacity(0.3)
p.fillRect(0, 0, xmax/10, xmax/10, QBrush(QColor(*col)))
p.setOpacity(1)
p.drawRect(0, 0, xmax/10, xmax/10)
p.translate(xmax/10, xmax/10)
p.scale(1, 1.5)
p.restore()
# p.scale(2, 2)
# p.rotate(45)
p.drawPixmap(0, 0, xmax/4, xmax/4, QPixmap(I('library.png')))
p.drawRect(0, 0, xmax/4, xmax/4)
f = p.font()
f.setPointSize(20)
# f.setLetterSpacing(f.PercentageSpacing, 200)
f.setUnderline(True)
# f.setOverline(True)
# f.setStrikeOut(True)
f.setFamily('Calibri')
p.setFont(f)
# p.setPen(QColor(0, 0, 255))
# p.scale(2, 2)
# p.rotate(45)
p.drawText(QPoint(xmax/3.9, 30), 'Some—text not By’s ū --- Д AV ff ff')
b = QBrush(Qt.HorPattern)
b.setColor(QColor(Qt.blue))
pix = QPixmap(I('console.png'))
w = xmax/4
p.fillRect(0, ymax/3, w, w, b)
p.fillRect(xmax/3, ymax/3, w, w, QBrush(pix))
x, y = 2*xmax/3, ymax/3
p.drawTiledPixmap(QRectF(x, y, w, w), pix, QPointF(10, 10))
x, y = 1, ymax/1.9
g = QLinearGradient(QPointF(x, y), QPointF(x+w, y+w))
g.setColorAt(0, QColor('#00f'))
g.setColorAt(1, QColor('#fff'))
p.fillRect(x, y, w, w, QBrush(g))
def run(dev, func):
p = QPainter(dev)
if isinstance(dev, PdfDevice):
dev.init_page()
xmax, ymax = p.viewport().width(), p.viewport().height()
try:
func(p, xmax, ymax)
finally:
p.end()
if isinstance(dev, PdfDevice):
if dev.engine.errors_occurred:
raise SystemExit(1)
def brush(p, xmax, ymax):
x = xmax/3
y = 0
w = xmax/2
pix = QPixmap(I('console.png'))
p.fillRect(x, y, w, w, QBrush(pix))
p.fillRect(0, y+xmax/1.9, w, w, QBrush(pix))
def pen(p, xmax, ymax):
pix = QPixmap(I('console.png'))
pen = QPen(QBrush(pix), 60)
p.setPen(pen)
p.drawRect(0, xmax/3, xmax/3, xmax/2)
def text(p, xmax, ymax):
p.drawText(QPoint(0, ymax/3), 'Text')
def main():
app = QApplication([])
app
tdir = gettempdir()
pdf = os.path.join(tdir, 'painter.pdf')
func = full
dpi = 100
with open(pdf, 'wb') as f:
dev = PdfDevice(f, xdpi=dpi, ydpi=dpi, compress=False)
img = QImage(dev.width(), dev.height(),
QImage.Format_ARGB32_Premultiplied)
img.setDotsPerMeterX(dpi*39.37)
img.setDotsPerMeterY(dpi*39.37)
img.fill(Qt.white)
run(dev, func)
run(img, func)
path = os.path.join(tdir, 'painter.png')
img.save(path)
print ('PDF written to:', pdf)
print ('Image written to:', path)
if __name__ == '__main__':
main()

7
src/calibre/gui2/convert/single.py
View File

@ -33,7 +33,10 @@ from calibre.utils.config import prefs
from calibre.utils.logging import Log from calibre.utils.logging import Log
class NoSupportedInputFormats(Exception): class NoSupportedInputFormats(Exception):
pass
def __init__(self, available_formats):
Exception.__init__(self)
self.available_formats = available_formats
def sort_formats_by_preference(formats, prefs): def sort_formats_by_preference(formats, prefs):
uprefs = [x.upper() for x in prefs] uprefs = [x.upper() for x in prefs]
@ -86,7 +89,7 @@ def get_supported_input_formats_for_book(db, book_id):
input_formats = set([x.lower() for x in supported_input_formats()]) input_formats = set([x.lower() for x in supported_input_formats()])
input_formats = sorted(available_formats.intersection(input_formats)) input_formats = sorted(available_formats.intersection(input_formats))
if not input_formats: if not input_formats:
raise NoSupportedInputFormats raise NoSupportedInputFormats(tuple(x for x in available_formats if x))
return input_formats return input_formats

2
src/calibre/gui2/custom_column_widgets.py
View File

@ -372,7 +372,7 @@ class Series(Base):
self.widgets.append(QLabel('&'+self.col_metadata['name']+_(' index:'), parent)) self.widgets.append(QLabel('&'+self.col_metadata['name']+_(' index:'), parent))
w = QDoubleSpinBox(parent) w = QDoubleSpinBox(parent)
w.setRange(-100., float(100000000)) w.setRange(-10000., float(100000000))
w.setDecimals(2) w.setDecimals(2)
w.setSingleStep(1) w.setSingleStep(1)
self.idx_widget=w self.idx_widget=w

6
src/calibre/gui2/library/models.py
View File

@ -5,7 +5,7 @@ __license__ = 'GPL v3'
__copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>' __copyright__ = '2010, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en' __docformat__ = 'restructuredtext en'
import functools, re, os, traceback, errno import functools, re, os, traceback, errno, time
from collections import defaultdict from collections import defaultdict
from PyQt4.Qt import (QAbstractTableModel, Qt, pyqtSignal, QIcon, QImage, from PyQt4.Qt import (QAbstractTableModel, Qt, pyqtSignal, QIcon, QImage,
@ -1419,7 +1419,11 @@ class DeviceBooksModel(BooksModel): # {{{
return QVariant(human_readable(size)) return QVariant(human_readable(size))
elif cname == 'timestamp': elif cname == 'timestamp':
dt = self.db[self.map[row]].datetime dt = self.db[self.map[row]].datetime
try:
dt = dt_factory(dt, assume_utc=True, as_utc=False) dt = dt_factory(dt, assume_utc=True, as_utc=False)
except OverflowError:
dt = dt_factory(time.gmtime(), assume_utc=True,
as_utc=False)
return QVariant(strftime(TIME_FMT, dt.timetuple())) return QVariant(strftime(TIME_FMT, dt.timetuple()))
elif cname == 'collections': elif cname == 'collections':
tags = self.db[self.map[row]].device_collections tags = self.db[self.map[row]].device_collections

3
src/calibre/gui2/metadata/basic_widgets.py
View File

@ -1094,6 +1094,9 @@ class RatingEdit(QSpinBox): # {{{
db.set_rating(id_, 2*self.current_val, notify=False, commit=False) db.set_rating(id_, 2*self.current_val, notify=False, commit=False)
return True return True
def zero(self):
self.setValue(0)
# }}} # }}}
class TagsEdit(EditWithComplete): # {{{ class TagsEdit(EditWithComplete): # {{{

14
src/calibre/gui2/metadata/single.py
View File

@ -181,6 +181,11 @@ class MetadataSingleDialogBase(ResizableDialog):
self.basic_metadata_widgets.append(self.comments) self.basic_metadata_widgets.append(self.comments)
self.rating = RatingEdit(self) self.rating = RatingEdit(self)
self.clear_ratings_button = QToolButton(self)
self.clear_ratings_button.setToolTip(_('Clear rating'))
self.clear_ratings_button.setIcon(QIcon(I('trash.png')))
self.clear_ratings_button.clicked.connect(self.rating.zero)
self.basic_metadata_widgets.append(self.rating) self.basic_metadata_widgets.append(self.rating)
self.tags = TagsEdit(self) self.tags = TagsEdit(self)
@ -659,8 +664,9 @@ class MetadataSingleDialog(MetadataSingleDialogBase): # {{{
QSizePolicy.Expanding) QSizePolicy.Expanding)
l.addItem(self.tabs[0].spc_one, 1, 0, 1, 3) l.addItem(self.tabs[0].spc_one, 1, 0, 1, 3)
sto(self.cover.buttons[-1], self.rating) sto(self.cover.buttons[-1], self.rating)
create_row2(1, self.rating) create_row2(1, self.rating, self.clear_ratings_button)
sto(self.rating, self.tags_editor_button) sto(self.rating, self.clear_ratings_button)
sto(self.clear_ratings_button, self.tags_editor_button)
sto(self.tags_editor_button, self.tags) sto(self.tags_editor_button, self.tags)
create_row2(2, self.tags, self.clear_tags_button, front_button=self.tags_editor_button) create_row2(2, self.tags, self.clear_tags_button, front_button=self.tags_editor_button)
sto(self.clear_tags_button, self.paste_isbn_button) sto(self.clear_tags_button, self.paste_isbn_button)
@ -780,7 +786,7 @@ class MetadataSingleDialogAlt1(MetadataSingleDialogBase): # {{{
button=self.clear_series_button, icon='trash.png') button=self.clear_series_button, icon='trash.png')
create_row(5, self.series_index, self.tags) create_row(5, self.series_index, self.tags)
create_row(6, self.tags, self.rating, button=self.clear_tags_button) create_row(6, self.tags, self.rating, button=self.clear_tags_button)
create_row(7, self.rating, self.pubdate) create_row(7, self.rating, self.pubdate, button=self.clear_ratings_button)
create_row(8, self.pubdate, self.publisher, create_row(8, self.pubdate, self.publisher,
button=self.pubdate.clear_button, icon='trash.png') button=self.pubdate.clear_button, icon='trash.png')
create_row(9, self.publisher, self.languages) create_row(9, self.publisher, self.languages)
@ -917,7 +923,7 @@ class MetadataSingleDialogAlt2(MetadataSingleDialogBase): # {{{
button=self.clear_series_button, icon='trash.png') button=self.clear_series_button, icon='trash.png')
create_row(5, self.series_index, self.tags) create_row(5, self.series_index, self.tags)
create_row(6, self.tags, self.rating, button=self.clear_tags_button) create_row(6, self.tags, self.rating, button=self.clear_tags_button)
create_row(7, self.rating, self.pubdate) create_row(7, self.rating, self.pubdate, button=self.clear_ratings_button)
create_row(8, self.pubdate, self.publisher, create_row(8, self.pubdate, self.publisher,
button=self.pubdate.clear_button, icon='trash.png') button=self.pubdate.clear_button, icon='trash.png')
create_row(9, self.publisher, self.languages) create_row(9, self.publisher, self.languages)

33
src/calibre/gui2/preferences/ignored_devices.py
View File

@ -7,8 +7,10 @@ __license__ = 'GPL v3'
__copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>' __copyright__ = '2012, Kovid Goyal <kovid at kovidgoyal.net>'
__docformat__ = 'restructuredtext en' __docformat__ = 'restructuredtext en'
from PyQt4.Qt import (QLabel, QVBoxLayout, QListWidget, QListWidgetItem, Qt) from PyQt4.Qt import (QLabel, QVBoxLayout, QListWidget, QListWidgetItem, Qt,
QIcon)
from calibre.customize.ui import enable_plugin
from calibre.gui2.preferences import ConfigWidgetBase, test_widget from calibre.gui2.preferences import ConfigWidgetBase, test_widget
class ConfigWidget(ConfigWidgetBase): class ConfigWidget(ConfigWidgetBase):
@ -31,6 +33,18 @@ class ConfigWidget(ConfigWidgetBase):
f.itemChanged.connect(self.changed_signal) f.itemChanged.connect(self.changed_signal)
f.itemDoubleClicked.connect(self.toggle_item) f.itemDoubleClicked.connect(self.toggle_item)
self.la2 = la = QLabel(_(
'The list of device plugins you have disabled. Uncheck an entry '
'to enable the plugin. calibre cannot detect devices that are '
'managed by disabled plugins.'))
la.setWordWrap(True)
l.addWidget(la)
self.device_plugins = f = QListWidget(f)
l.addWidget(f)
f.itemChanged.connect(self.changed_signal)
f.itemDoubleClicked.connect(self.toggle_item)
def toggle_item(self, item): def toggle_item(self, item):
item.setCheckState(Qt.Checked if item.checkState() == Qt.Unchecked else item.setCheckState(Qt.Checked if item.checkState() == Qt.Unchecked else
Qt.Unchecked) Qt.Unchecked)
@ -46,6 +60,17 @@ class ConfigWidget(ConfigWidgetBase):
item.setCheckState(Qt.Checked) item.setCheckState(Qt.Checked)
self.devices.blockSignals(False) self.devices.blockSignals(False)
self.device_plugins.blockSignals(True)
for dev in self.gui.device_manager.disabled_device_plugins:
n = dev.get_gui_name()
item = QListWidgetItem(n, self.device_plugins)
item.setData(Qt.UserRole, dev)
item.setFlags(Qt.ItemIsEnabled|Qt.ItemIsUserCheckable|Qt.ItemIsSelectable)
item.setCheckState(Qt.Checked)
item.setIcon(QIcon(I('plugins.png')))
self.device_plugins.sortItems()
self.device_plugins.blockSignals(False)
def restore_defaults(self): def restore_defaults(self):
if self.devices.count() > 0: if self.devices.count() > 0:
self.devices.clear() self.devices.clear()
@ -63,6 +88,12 @@ class ConfigWidget(ConfigWidgetBase):
for dev, bl in devs.iteritems(): for dev, bl in devs.iteritems():
dev.set_user_blacklisted_devices(bl) dev.set_user_blacklisted_devices(bl)
for i in xrange(self.device_plugins.count()):
e = self.device_plugins.item(i)
dev = e.data(Qt.UserRole).toPyObject()
if e.checkState() == Qt.Unchecked:
enable_plugin(dev)
return True # Restart required return True # Restart required
if __name__ == '__main__': if __name__ == '__main__':

2
src/calibre/gui2/preferences/server.ui
View File

@ -273,7 +273,7 @@
<widget class="QLabel" name="label_13"> <widget class="QLabel" name="label_13">
<property name="text"> <property name="text">
<string>&lt;p&gt;Remember to leave calibre running as the server only runs as long as calibre is running. <string>&lt;p&gt;Remember to leave calibre running as the server only runs as long as calibre is running.
&lt;p&gt;To connect to the calibre server from your device you should use a URL of the form &lt;b&gt;http://myhostname:8080&lt;/b&gt; as a new catalog in the Stanza reader on your iPhone. Here myhostname should be either the fully qualified hostname or the IP address of the computer calibre is running on.</string> &lt;p&gt;To connect to the calibre server from your device you should use a URL of the form &lt;b&gt;http://myhostname:8080&lt;/b&gt;. Here myhostname should be either the fully qualified hostname or the IP address of the computer calibre is running on. If you want to access the server from anywhere in the world, you will have to setup port forwarding for it on your router.</string>
</property> </property>
<property name="wordWrap"> <property name="wordWrap">
<bool>true</bool> <bool>true</bool>

12
src/calibre/gui2/store/stores/bn_plugin.py
View File

@ -6,6 +6,7 @@ __license__ = 'GPL 3'
__copyright__ = '2011, John Schember <john@nachtimwald.com>' __copyright__ = '2011, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en' __docformat__ = 'restructuredtext en'
import re
import urllib import urllib
from contextlib import closing from contextlib import closing
@ -50,12 +51,17 @@ class BNStore(BasicStoreConfig, StorePlugin):
if not id: if not id:
continue continue
cover_url = ''.join(data.xpath('.//img[contains(@class, "product-image")]/@src')) cover_url = ''
cover_id = ''.join(data.xpath('.//img[contains(@class, "product-image")]/@id'))
m = re.search(r"%s'.*?srcUrl: '(?P<iurl>.*?)'.*?}" % cover_id, raw)
if m:
cover_url = m.group('iurl')
title = ''.join(data.xpath('descendant::p[@class="title"]//span[@class="name"]//text()')).strip() title = ''.join(data.xpath('descendant::p[@class="title"]//span[@class="name"]//text()')).strip()
if not title: continue if not title:
continue
author = ', '.join(data.xpath('.//ul[@class="contributors"]//a[@class="subtle"]//text()')).strip() author = ', '.join(data.xpath('.//ul[contains(@class, "contributors")]//a[contains(@class, "subtle")]//text()')).strip()
price = ''.join(data.xpath('.//a[contains(@class, "bn-price")]//text()')) price = ''.join(data.xpath('.//a[contains(@class, "bn-price")]//text()'))
counter -= 1 counter -= 1

4
src/calibre/gui2/store/stores/google_books_plugin.py
View File

@ -59,7 +59,7 @@ class GoogleBooksStore(BasicStoreConfig, StorePlugin):
counter = max_results counter = max_results
with closing(br.open(url, timeout=timeout)) as f: with closing(br.open(url, timeout=timeout)) as f:
doc = html.fromstring(f.read()) doc = html.fromstring(f.read())
for data in doc.xpath('//ol[@id="rso"]/li'): for data in doc.xpath('//ol/li'):
if counter <= 0: if counter <= 0:
break break
@ -68,7 +68,7 @@ class GoogleBooksStore(BasicStoreConfig, StorePlugin):
continue continue
title = ''.join(data.xpath('.//h3/a//text()')) title = ''.join(data.xpath('.//h3/a//text()'))
authors = data.xpath('.//div[@class="f"]//a//text()') authors = data.xpath('.//span[contains(@class, "f")]//a//text()')
while authors and authors[-1].strip().lower() in ('preview', 'read', 'more editions'): while authors and authors[-1].strip().lower() in ('preview', 'read', 'more editions'):
authors = authors[:-1] authors = authors[:-1]
if not authors: if not authors:

75
src/calibre/gui2/store/stores/nook_uk_plugin.py Normal file
View File

@ -0,0 +1,75 @@
# -*- coding: utf-8 -*-
from __future__ import (unicode_literals, division, absolute_import, print_function)
__license__ = 'GPL 3'
__copyright__ = '2012, John Schember <john@nachtimwald.com>'
__docformat__ = 'restructuredtext en'
import re
import urllib
from contextlib import closing
from lxml import html
from PyQt4.Qt import QUrl
from calibre import browser, url_slash_cleaner
from calibre.gui2 import open_url
from calibre.gui2.store import StorePlugin
from calibre.gui2.store.basic_config import BasicStoreConfig
from calibre.gui2.store.search_result import SearchResult
from calibre.gui2.store.web_store_dialog import WebStoreDialog
class NookUKStore(BasicStoreConfig, StorePlugin):
def open(self, parent=None, detail_item=None, external=False):
url = "http://uk.nook.com"
if external or self.config.get('open_external', False):
open_url(QUrl(url_slash_cleaner(detail_item if detail_item else url)))
else:
d = WebStoreDialog(self.gui, url, parent, detail_item)
d.setWindowTitle(self.name)
d.set_tags(self.config.get('tags', ''))
d.exec_()
def search(self, query, max_results=10, timeout=60):
url = u'http://uk.nook.com/s/%s?s%%5Bdref%%5D=1&s%%5Bkeyword%%5D=%s' % (query.replace(' ', '-'), urllib.quote(query))
br = browser()
counter = max_results
with closing(br.open(url, timeout=timeout)) as f:
raw = f.read()
doc = html.fromstring(raw)
for data in doc.xpath('//ul[contains(@class, "product_list")]/li'):
if counter <= 0:
break
id = ''.join(data.xpath('.//span[contains(@class, "image")]/a/@href'))
if not id:
continue
cover_url = ''.join(data.xpath('.//span[contains(@class, "image")]//img/@data-src'))
title = ''.join(data.xpath('.//div[contains(@class, "title")]//text()')).strip()
if not title:
continue
author = ', '.join(data.xpath('.//div[contains(@class, "contributor")]//a/text()')).strip()
price = ''.join(data.xpath('.//div[contains(@class, "action")]//a//text()')).strip()
price = re.sub(r'[^\d.,£]', '', price);
counter -= 1
s = SearchResult()
s.cover_url = cover_url
s.title = title.strip()
s.author = author.strip()
s.price = price.strip()
s.detail_item = 'http://uk.nook.com/' + id.strip()
s.drm = SearchResult.DRM_UNKNOWN
s.formats = 'Nook'
yield s

2
src/calibre/gui2/store/stores/smashwords_plugin.py
View File

@ -76,7 +76,7 @@ class SmashwordsStore(BasicStoreConfig, StorePlugin):
title = ''.join(data.xpath('//a[@class="bookTitle"]/text()')) title = ''.join(data.xpath('//a[@class="bookTitle"]/text()'))
subnote = ''.join(data.xpath('//span[@class="subnote"]/text()')) subnote = ''.join(data.xpath('//span[@class="subnote"]/text()'))
author = ''.join(data.xpath('//span[@class="subnote"]/a/text()')) author = ''.join(data.xpath('//span[@class="subnote"]//a[1]//text()'))
if '$' in subnote: if '$' in subnote:
price = subnote.partition('$')[2] price = subnote.partition('$')[2]
price = price.split(u'\xa0')[0] price = price.split(u'\xa0')[0]

27
src/calibre/gui2/tools.py
View File

@ -88,19 +88,34 @@ def convert_single_ebook(parent, db, book_ids, auto_conversion=False, # {{{
changed = True changed = True
d.break_cycles() d.break_cycles()
except NoSupportedInputFormats: except NoSupportedInputFormats as nsif:
bad.append(book_id) bad.append((book_id, nsif.available_formats))
if bad and show_no_format_warning: if bad and show_no_format_warning:
if len(bad) == 1 and not bad[0][1]:
title = db.title(bad[0][0], True)
warning_dialog(parent, _('Could not convert'), '<p>'+
_('Could not convert <b>%s</b> as it has no ebook files. If you '
'think it should have files, but calibre is not finding '
'them, that is most likely because you moved the book\'s '
'files around outside of calibre. You will need to find those files '
'and re-add them to calibre.')%title, show=True)
else:
res = [] res = []
for id in bad: for id, available_formats in bad:
title = db.title(id, True) title = db.title(id, True)
res.append('%s'%title) if available_formats:
msg = _('No supported formats (Available formats: %s)')%(
', '.join(available_formats))
else:
msg = _('This book has no actual ebook files')
res.append('%s - %s'%(title, msg))
msg = '%s' % '\n'.join(res) msg = '%s' % '\n'.join(res)
warning_dialog(parent, _('Could not convert some books'), warning_dialog(parent, _('Could not convert some books'),
_('Could not convert %(num)d of %(tot)d books, because no suitable source' _('Could not convert %(num)d of %(tot)d books, because no supported source'
' format was found.') % dict(num=len(res), tot=total), ' formats were found.') % dict(num=len(res), tot=total),
msg).exec_() msg).exec_()
return jobs, changed, bad return jobs, changed, bad

2
src/calibre/utils/filenames.py
View File

@ -17,7 +17,7 @@ def ascii_text(orig):
ascii = udc.decode(orig) ascii = udc.decode(orig)
except: except:
if isinstance(orig, unicode): if isinstance(orig, unicode):
ascii = orig.encode('ascii', 'replace') orig = orig.encode('ascii', 'replace')
ascii = orig.decode(preferred_encoding, ascii = orig.decode(preferred_encoding,
'replace').encode('ascii', 'replace') 'replace').encode('ascii', 'replace')
return ascii return ascii

2
src/calibre/utils/fonts/sfnt/subset.py
View File

@ -84,7 +84,7 @@ def do_warn(warnings, *args):
def pdf_subset(sfnt, glyphs): def pdf_subset(sfnt, glyphs):
for tag in tuple(sfnt.tables): for tag in tuple(sfnt.tables):
if tag not in {b'hhea', b'head', b'hmtx', b'maxp', if tag not in {b'hhea', b'head', b'hmtx', b'maxp',
b'OS/2', b'post', b'cvt', b'fpgm', b'glyf', b'loca', b'OS/2', b'post', b'cvt ', b'fpgm', b'glyf', b'loca',
b'prep', b'CFF ', b'VORG'}: b'prep', b'CFF ', b'VORG'}:
# Remove non core tables since they are unused in PDF rendering # Remove non core tables since they are unused in PDF rendering
del sfnt[tag] del sfnt[tag]

248
src/calibre/utils/pyparsing.py
View File

@ -58,8 +58,8 @@ The pyparsing module handles some of the problems that are typically vexing when
- embedded comments - embedded comments
""" """
__version__ = "1.5.6" __version__ = "1.5.7"
__versionTime__ = "26 June 2011 10:53" __versionTime__ = "17 November 2012 16:18"
__author__ = "Paul McGuire <ptmcg@users.sourceforge.net>" __author__ = "Paul McGuire <ptmcg@users.sourceforge.net>"
import string import string
@ -81,34 +81,21 @@ __all__ = [
'White', 'Word', 'WordEnd', 'WordStart', 'ZeroOrMore', 'White', 'Word', 'WordEnd', 'WordStart', 'ZeroOrMore',
'alphanums', 'alphas', 'alphas8bit', 'anyCloseTag', 'anyOpenTag', 'cStyleComment', 'col', 'alphanums', 'alphas', 'alphas8bit', 'anyCloseTag', 'anyOpenTag', 'cStyleComment', 'col',
'commaSeparatedList', 'commonHTMLEntity', 'countedArray', 'cppStyleComment', 'dblQuotedString', 'commaSeparatedList', 'commonHTMLEntity', 'countedArray', 'cppStyleComment', 'dblQuotedString',
'dblSlashComment', 'delimitedList', 'dictOf', 'downcaseTokens', 'empty', 'getTokensEndLoc', 'hexnums', 'dblSlashComment', 'delimitedList', 'dictOf', 'downcaseTokens', 'empty', 'hexnums',
'htmlComment', 'javaStyleComment', 'keepOriginalText', 'line', 'lineEnd', 'lineStart', 'lineno', 'htmlComment', 'javaStyleComment', 'keepOriginalText', 'line', 'lineEnd', 'lineStart', 'lineno',
'makeHTMLTags', 'makeXMLTags', 'matchOnlyAtCol', 'matchPreviousExpr', 'matchPreviousLiteral', 'makeHTMLTags', 'makeXMLTags', 'matchOnlyAtCol', 'matchPreviousExpr', 'matchPreviousLiteral',
'nestedExpr', 'nullDebugAction', 'nums', 'oneOf', 'opAssoc', 'operatorPrecedence', 'printables', 'nestedExpr', 'nullDebugAction', 'nums', 'oneOf', 'opAssoc', 'operatorPrecedence', 'printables',
'punc8bit', 'pythonStyleComment', 'quotedString', 'removeQuotes', 'replaceHTMLEntity', 'punc8bit', 'pythonStyleComment', 'quotedString', 'removeQuotes', 'replaceHTMLEntity',
'replaceWith', 'restOfLine', 'sglQuotedString', 'srange', 'stringEnd', 'replaceWith', 'restOfLine', 'sglQuotedString', 'srange', 'stringEnd',
'stringStart', 'traceParseAction', 'unicodeString', 'upcaseTokens', 'withAttribute', 'stringStart', 'traceParseAction', 'unicodeString', 'upcaseTokens', 'withAttribute',
'indentedBlock', 'originalTextFor', 'indentedBlock', 'originalTextFor', 'ungroup', 'infixNotation',
] ]
""" _MAX_INT = sys.maxint
Detect if we are running version 3.X and make appropriate changes range = xrange
Robert A. Clark set = lambda s : dict( [(c,0) for c in s] )
"""
_PY3K = sys.version_info[0] > 2
if _PY3K:
_MAX_INT = sys.maxsize
basestring = str
unichr = chr
_ustr = str
alphas = string.ascii_lowercase + string.ascii_uppercase
else:
_MAX_INT = sys.maxint
range = xrange
set = lambda s : dict( [(c,0) for c in s] )
alphas = string.lowercase + string.uppercase
def _ustr(obj): def _ustr(obj):
"""Drop-in replacement for str(obj) that tries to be Unicode friendly. It first tries """Drop-in replacement for str(obj) that tries to be Unicode friendly. It first tries
str(obj). If that fails with a UnicodeEncodeError, then it tries unicode(obj). It str(obj). If that fails with a UnicodeEncodeError, then it tries unicode(obj). It
then < returns the unicode object | encodes it with the default encoding | ... >. then < returns the unicode object | encodes it with the default encoding | ... >.
@ -135,12 +122,10 @@ else:
#return unicode(obj).encode(sys.getdefaultencoding(), 'replace') #return unicode(obj).encode(sys.getdefaultencoding(), 'replace')
# ... # ...
alphas = string.lowercase + string.uppercase
# build list of single arg builtins, tolerant of Python version, that can be used as parse actions # build list of single arg builtins, tolerant of Python version, that can be used as parse actions
singleArgBuiltins = [] singleArgBuiltins = []
import __builtin__ import __builtin__
for fname in "sum len enumerate sorted reversed list tuple set any all".split(): for fname in "sum len sorted reversed list tuple set any all min max".split():
try: try:
singleArgBuiltins.append(getattr(__builtin__,fname)) singleArgBuiltins.append(getattr(__builtin__,fname))
except AttributeError: except AttributeError:
@ -159,7 +144,8 @@ def _xml_escape(data):
class _Constants(object): class _Constants(object):
pass pass
nums = string.digits alphas = string.ascii_lowercase + string.ascii_uppercase
nums = "0123456789"
hexnums = nums + "ABCDEFabcdef" hexnums = nums + "ABCDEFabcdef"
alphanums = alphas + nums alphanums = alphas + nums
_bslash = chr(92) _bslash = chr(92)
@ -211,7 +197,7 @@ class ParseBaseException(Exception):
return line_str.strip() return line_str.strip()
def __dir__(self): def __dir__(self):
return "loc msg pstr parserElement lineno col line " \ return "loc msg pstr parserElement lineno col line " \
"markInputLine __str__ __repr__".split() "markInputline __str__ __repr__".split()
class ParseException(ParseBaseException): class ParseException(ParseBaseException):
"""exception thrown when parse expressions don't match class; """exception thrown when parse expressions don't match class;
@ -228,8 +214,8 @@ class ParseFatalException(ParseBaseException):
pass pass
class ParseSyntaxException(ParseFatalException): class ParseSyntaxException(ParseFatalException):
"""just like C{ParseFatalException}, but thrown internally when an """just like C{L{ParseFatalException}}, but thrown internally when an
C{ErrorStop} ('-' operator) indicates that parsing is to stop immediately because C{L{ErrorStop<And._ErrorStop>}} ('-' operator) indicates that parsing is to stop immediately because
an unbacktrackable syntax error has been found""" an unbacktrackable syntax error has been found"""
def __init__(self, pe): def __init__(self, pe):
super(ParseSyntaxException, self).__init__( super(ParseSyntaxException, self).__init__(
@ -444,16 +430,13 @@ class ParseResults(object):
return "(%s, %s)" % ( repr( self.__toklist ), repr( self.__tokdict ) ) return "(%s, %s)" % ( repr( self.__toklist ), repr( self.__tokdict ) )
def __str__( self ): def __str__( self ):
out = "[" out = []
sep = ""
for i in self.__toklist: for i in self.__toklist:
if isinstance(i, ParseResults): if isinstance(i, ParseResults):
out += sep + _ustr(i) out.append(_ustr(i))
else: else:
out += sep + repr(i) out.append(repr(i))
sep = ", " return '[' + ', '.join(out) + ']'
out += "]"
return out
def _asStringList( self, sep='' ): def _asStringList( self, sep='' ):
out = [] out = []
@ -616,7 +599,7 @@ class ParseResults(object):
self.__parent = None self.__parent = None
def __dir__(self): def __dir__(self):
return dir(super(ParseResults,self)) + self.keys() return dir(super(ParseResults,self)) + list(self.keys())
def col (loc,strg): def col (loc,strg):
"""Returns current column within a string, counting newlines as line separators. """Returns current column within a string, counting newlines as line separators.
@ -624,7 +607,7 @@ def col (loc,strg):
Note: the default parsing behavior is to expand tabs in the input string Note: the default parsing behavior is to expand tabs in the input string
before starting the parsing process. See L{I{ParserElement.parseString}<ParserElement.parseString>} for more information before starting the parsing process. See L{I{ParserElement.parseString}<ParserElement.parseString>} for more information
on parsing strings containing <TAB>s, and suggested methods to maintain a on parsing strings containing C{<TAB>}s, and suggested methods to maintain a
consistent view of the parsed string, the parse location, and line and column consistent view of the parsed string, the parse location, and line and column
positions within the parsed string. positions within the parsed string.
""" """
@ -636,7 +619,7 @@ def lineno(loc,strg):
Note: the default parsing behavior is to expand tabs in the input string Note: the default parsing behavior is to expand tabs in the input string
before starting the parsing process. See L{I{ParserElement.parseString}<ParserElement.parseString>} for more information before starting the parsing process. See L{I{ParserElement.parseString}<ParserElement.parseString>} for more information
on parsing strings containing <TAB>s, and suggested methods to maintain a on parsing strings containing C{<TAB>}s, and suggested methods to maintain a
consistent view of the parsed string, the parse location, and line and column consistent view of the parsed string, the parse location, and line and column
positions within the parsed string. positions within the parsed string.
""" """
@ -666,33 +649,23 @@ def nullDebugAction(*args):
pass pass
'decorator to trim function calls to match the arity of the target' 'decorator to trim function calls to match the arity of the target'
if not _PY3K: def _trim_arity(func, maxargs=2):
def _trim_arity(func, maxargs=2): if func in singleArgBuiltins:
return lambda s,l,t: func(t)
limit = [0] limit = [0]
foundArity = [False]
def wrapper(*args): def wrapper(*args):
while 1: while 1:
try: try:
return func(*args[limit[0]:]) ret = func(*args[limit[0]:])
foundArity[0] = True
return ret
except TypeError: except TypeError:
if limit[0] <= maxargs: if limit[0] <= maxargs and not foundArity[0]:
limit[0] += 1 limit[0] += 1
continue continue
raise raise
return wrapper return wrapper
else:
def _trim_arity(func, maxargs=2):
limit = maxargs
def wrapper(*args):
#~ nonlocal limit
while 1:
try:
return func(*args[limit:])
except TypeError:
if limit:
limit -= 1
continue
raise
return wrapper
class ParserElement(object): class ParserElement(object):
"""Abstract base level parser element class.""" """Abstract base level parser element class."""
@ -705,6 +678,13 @@ class ParserElement(object):
ParserElement.DEFAULT_WHITE_CHARS = chars ParserElement.DEFAULT_WHITE_CHARS = chars
setDefaultWhitespaceChars = staticmethod(setDefaultWhitespaceChars) setDefaultWhitespaceChars = staticmethod(setDefaultWhitespaceChars)
def inlineLiteralsUsing(cls):
"""
Set class to be used for inclusion of string literals into a parser.
"""
ParserElement.literalStringClass = cls
inlineLiteralsUsing = staticmethod(inlineLiteralsUsing)
def __init__( self, savelist=False ): def __init__( self, savelist=False ):
self.parseAction = list() self.parseAction = list()
self.failAction = None self.failAction = None
@ -789,14 +769,14 @@ class ParserElement(object):
C{fn(loc,toks)}, C{fn(toks)}, or just C{fn()}, where: C{fn(loc,toks)}, C{fn(toks)}, or just C{fn()}, where:
- s = the original string being parsed (see note below) - s = the original string being parsed (see note below)
- loc = the location of the matching substring - loc = the location of the matching substring
- toks = a list of the matched tokens, packaged as a ParseResults object - toks = a list of the matched tokens, packaged as a C{L{ParseResults}} object
If the functions in fns modify the tokens, they can return them as the return If the functions in fns modify the tokens, they can return them as the return
value from fn, and the modified list of tokens will replace the original. value from fn, and the modified list of tokens will replace the original.
Otherwise, fn does not need to return any value. Otherwise, fn does not need to return any value.
Note: the default parsing behavior is to expand tabs in the input string Note: the default parsing behavior is to expand tabs in the input string
before starting the parsing process. See L{I{parseString}<parseString>} for more information before starting the parsing process. See L{I{parseString}<parseString>} for more information
on parsing strings containing <TAB>s, and suggested methods to maintain a on parsing strings containing C{<TAB>}s, and suggested methods to maintain a
consistent view of the parsed string, the parse location, and line and column consistent view of the parsed string, the parse location, and line and column
positions within the parsed string. positions within the parsed string.
""" """
@ -818,7 +798,7 @@ class ParserElement(object):
- loc = location where expression match was attempted and failed - loc = location where expression match was attempted and failed
- expr = the parse expression that failed - expr = the parse expression that failed
- err = the exception thrown - err = the exception thrown
The function returns no value. It may throw C{ParseFatalException} The function returns no value. It may throw C{L{ParseFatalException}}
if it is desired to stop parsing immediately.""" if it is desired to stop parsing immediately."""
self.failAction = fn self.failAction = fn
return self return self
@ -872,15 +852,12 @@ class ParserElement(object):
loc,tokens = self.parseImpl( instring, preloc, doActions ) loc,tokens = self.parseImpl( instring, preloc, doActions )
except IndexError: except IndexError:
raise ParseException( instring, len(instring), self.errmsg, self ) raise ParseException( instring, len(instring), self.errmsg, self )
except ParseBaseException: except ParseBaseException, err:
#~ print ("Exception raised:", err) #~ print ("Exception raised:", err)
err = None err = None
if self.debugActions[2]: if self.debugActions[2]:
err = sys.exc_info()[1]
self.debugActions[2]( instring, tokensStart, self, err ) self.debugActions[2]( instring, tokensStart, self, err )
if self.failAction: if self.failAction:
if err is None:
err = sys.exc_info()[1]
self.failAction( instring, tokensStart, self, err ) self.failAction( instring, tokensStart, self, err )
raise raise
else: else:
@ -910,10 +887,9 @@ class ParserElement(object):
self.resultsName, self.resultsName,
asList=self.saveAsList and isinstance(tokens,(ParseResults,list)), asList=self.saveAsList and isinstance(tokens,(ParseResults,list)),
modal=self.modalResults ) modal=self.modalResults )
except ParseBaseException: except ParseBaseException, err:
#~ print "Exception raised in user parse action:", err #~ print "Exception raised in user parse action:", err
if (self.debugActions[2] ): if (self.debugActions[2] ):
err = sys.exc_info()[1]
self.debugActions[2]( instring, tokensStart, self, err ) self.debugActions[2]( instring, tokensStart, self, err )
raise raise
else: else:
@ -952,8 +928,7 @@ class ParserElement(object):
value = self._parseNoCache( instring, loc, doActions, callPreParse ) value = self._parseNoCache( instring, loc, doActions, callPreParse )
ParserElement._exprArgCache[ lookup ] = (value[0],value[1].copy()) ParserElement._exprArgCache[ lookup ] = (value[0],value[1].copy())
return value return value
except ParseBaseException: except ParseBaseException, pe:
pe = sys.exc_info()[1]
ParserElement._exprArgCache[ lookup ] = pe ParserElement._exprArgCache[ lookup ] = pe
raise raise
@ -994,7 +969,7 @@ class ParserElement(object):
If you want the grammar to require that the entire input string be If you want the grammar to require that the entire input string be
successfully parsed, then set C{parseAll} to True (equivalent to ending successfully parsed, then set C{parseAll} to True (equivalent to ending
the grammar with C{StringEnd()}). the grammar with C{L{StringEnd()}}).
Note: C{parseString} implicitly calls C{expandtabs()} on the input string, Note: C{parseString} implicitly calls C{expandtabs()} on the input string,
in order to report proper column numbers in parse actions. in order to report proper column numbers in parse actions.
@ -1023,12 +998,11 @@ class ParserElement(object):
loc = self.preParse( instring, loc ) loc = self.preParse( instring, loc )
se = Empty() + StringEnd() se = Empty() + StringEnd()
se._parse( instring, loc ) se._parse( instring, loc )
except ParseBaseException: except ParseBaseException, exc:
if ParserElement.verbose_stacktrace: if ParserElement.verbose_stacktrace:
raise raise
else: else:
# catch and re-raise exception from here, clears out pyparsing internal stack trace # catch and re-raise exception from here, clears out pyparsing internal stack trace
exc = sys.exc_info()[1]
raise exc raise exc
else: else:
return tokens return tokens
@ -1076,16 +1050,15 @@ class ParserElement(object):
loc = nextLoc loc = nextLoc
else: else:
loc = preloc+1 loc = preloc+1
except ParseBaseException: except ParseBaseException, exc:
if ParserElement.verbose_stacktrace: if ParserElement.verbose_stacktrace:
raise raise
else: else:
# catch and re-raise exception from here, clears out pyparsing internal stack trace # catch and re-raise exception from here, clears out pyparsing internal stack trace
exc = sys.exc_info()[1]
raise exc raise exc
def transformString( self, instring ): def transformString( self, instring ):
"""Extension to C{scanString}, to modify matching text with modified tokens that may """Extension to C{L{scanString}}, to modify matching text with modified tokens that may
be returned from a parse action. To use C{transformString}, define a grammar and be returned from a parse action. To use C{transformString}, define a grammar and
attach a parse action to it that modifies the returned token list. attach a parse action to it that modifies the returned token list.
Invoking C{transformString()} on a target string will then scan for matches, Invoking C{transformString()} on a target string will then scan for matches,
@ -1110,33 +1083,31 @@ class ParserElement(object):
out.append(instring[lastE:]) out.append(instring[lastE:])
out = [o for o in out if o] out = [o for o in out if o]
return "".join(map(_ustr,_flatten(out))) return "".join(map(_ustr,_flatten(out)))
except ParseBaseException: except ParseBaseException, exc:
if ParserElement.verbose_stacktrace: if ParserElement.verbose_stacktrace:
raise raise
else: else:
# catch and re-raise exception from here, clears out pyparsing internal stack trace # catch and re-raise exception from here, clears out pyparsing internal stack trace
exc = sys.exc_info()[1]
raise exc raise exc
def searchString( self, instring, maxMatches=_MAX_INT ): def searchString( self, instring, maxMatches=_MAX_INT ):
"""Another extension to C{scanString}, simplifying the access to the tokens found """Another extension to C{L{scanString}}, simplifying the access to the tokens found
to match the given parse expression. May be called with optional to match the given parse expression. May be called with optional
C{maxMatches} argument, to clip searching after 'n' matches are found. C{maxMatches} argument, to clip searching after 'n' matches are found.
""" """
try: try:
return ParseResults([ t for t,s,e in self.scanString( instring, maxMatches ) ]) return ParseResults([ t for t,s,e in self.scanString( instring, maxMatches ) ])
except ParseBaseException: except ParseBaseException, exc:
if ParserElement.verbose_stacktrace: if ParserElement.verbose_stacktrace:
raise raise
else: else:
# catch and re-raise exception from here, clears out pyparsing internal stack trace # catch and re-raise exception from here, clears out pyparsing internal stack trace
exc = sys.exc_info()[1]
raise exc raise exc
def __add__(self, other ): def __add__(self, other ):
"""Implementation of + operator - returns And""" """Implementation of + operator - returns C{L{And}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1144,9 +1115,9 @@ class ParserElement(object):
return And( [ self, other ] ) return And( [ self, other ] )
def __radd__(self, other ): def __radd__(self, other ):
"""Implementation of + operator when left operand is not a C{ParserElement}""" """Implementation of + operator when left operand is not a C{L{ParserElement}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1154,9 +1125,9 @@ class ParserElement(object):
return other + self return other + self
def __sub__(self, other): def __sub__(self, other):
"""Implementation of - operator, returns C{And} with error stop""" """Implementation of - operator, returns C{L{And}} with error stop"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1164,9 +1135,9 @@ class ParserElement(object):
return And( [ self, And._ErrorStop(), other ] ) return And( [ self, And._ErrorStop(), other ] )
def __rsub__(self, other ): def __rsub__(self, other ):
"""Implementation of - operator when left operand is not a C{ParserElement}""" """Implementation of - operator when left operand is not a C{L{ParserElement}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1179,12 +1150,12 @@ class ParserElement(object):
tuple, similar to C{{min,max}} multipliers in regular expressions. Tuples tuple, similar to C{{min,max}} multipliers in regular expressions. Tuples
may also include C{None} as in: may also include C{None} as in:
- C{expr*(n,None)} or C{expr*(n,)} is equivalent - C{expr*(n,None)} or C{expr*(n,)} is equivalent
to C{expr*n + ZeroOrMore(expr)} to C{expr*n + L{ZeroOrMore}(expr)}
(read as "at least n instances of C{expr}") (read as "at least n instances of C{expr}")
- C{expr*(None,n)} is equivalent to C{expr*(0,n)} - C{expr*(None,n)} is equivalent to C{expr*(0,n)}
(read as "0 to n instances of C{expr}") (read as "0 to n instances of C{expr}")
- C{expr*(None,None)} is equivalent to C{ZeroOrMore(expr)} - C{expr*(None,None)} is equivalent to C{L{ZeroOrMore}(expr)}
- C{expr*(1,None)} is equivalent to C{OneOrMore(expr)} - C{expr*(1,None)} is equivalent to C{L{OneOrMore}(expr)}
Note that C{expr*(None,n)} does not raise an exception if Note that C{expr*(None,n)} does not raise an exception if
more than n exprs exist in the input stream; that is, more than n exprs exist in the input stream; that is,
@ -1245,9 +1216,9 @@ class ParserElement(object):
return self.__mul__(other) return self.__mul__(other)
def __or__(self, other ): def __or__(self, other ):
"""Implementation of | operator - returns C{MatchFirst}""" """Implementation of | operator - returns C{L{MatchFirst}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1255,9 +1226,9 @@ class ParserElement(object):
return MatchFirst( [ self, other ] ) return MatchFirst( [ self, other ] )
def __ror__(self, other ): def __ror__(self, other ):
"""Implementation of | operator when left operand is not a C{ParserElement}""" """Implementation of | operator when left operand is not a C{L{ParserElement}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1265,9 +1236,9 @@ class ParserElement(object):
return other | self return other | self
def __xor__(self, other ): def __xor__(self, other ):
"""Implementation of ^ operator - returns C{Or}""" """Implementation of ^ operator - returns C{L{Or}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1275,9 +1246,9 @@ class ParserElement(object):
return Or( [ self, other ] ) return Or( [ self, other ] )
def __rxor__(self, other ): def __rxor__(self, other ):
"""Implementation of ^ operator when left operand is not a C{ParserElement}""" """Implementation of ^ operator when left operand is not a C{L{ParserElement}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1285,9 +1256,9 @@ class ParserElement(object):
return other ^ self return other ^ self
def __and__(self, other ): def __and__(self, other ):
"""Implementation of & operator - returns C{Each}""" """Implementation of & operator - returns C{L{Each}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1295,9 +1266,9 @@ class ParserElement(object):
return Each( [ self, other ] ) return Each( [ self, other ] )
def __rand__(self, other ): def __rand__(self, other ):
"""Implementation of & operator when left operand is not a C{ParserElement}""" """Implementation of & operator when left operand is not a C{L{ParserElement}}"""
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
if not isinstance( other, ParserElement ): if not isinstance( other, ParserElement ):
warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
SyntaxWarning, stacklevel=2) SyntaxWarning, stacklevel=2)
@ -1305,11 +1276,11 @@ class ParserElement(object):
return other & self return other & self
def __invert__( self ): def __invert__( self ):
"""Implementation of ~ operator - returns C{NotAny}""" """Implementation of ~ operator - returns C{L{NotAny}}"""
return NotAny( self ) return NotAny( self )
def __call__(self, name): def __call__(self, name):
"""Shortcut for C{setResultsName}, with C{listAllMatches=default}:: """Shortcut for C{L{setResultsName}}, with C{listAllMatches=default}::
userdata = Word(alphas).setResultsName("name") + Word(nums+"-").setResultsName("socsecno") userdata = Word(alphas).setResultsName("name") + Word(nums+"-").setResultsName("socsecno")
could be written as:: could be written as::
userdata = Word(alphas)("name") + Word(nums+"-")("socsecno") userdata = Word(alphas)("name") + Word(nums+"-")("socsecno")
@ -1403,14 +1374,16 @@ class ParserElement(object):
try: try:
file_contents = file_or_filename.read() file_contents = file_or_filename.read()
except AttributeError: except AttributeError:
f = open(file_or_filename, "rb") f = open(file_or_filename, "r")
file_contents = f.read() file_contents = f.read()
f.close() f.close()
try: try:
return self.parseString(file_contents, parseAll) return self.parseString(file_contents, parseAll)
except ParseBaseException: except ParseBaseException, exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clears out pyparsing internal stack trace # catch and re-raise exception from here, clears out pyparsing internal stack trace
exc = sys.exc_info()[1]
raise exc raise exc
def getException(self): def getException(self):
@ -1515,10 +1488,11 @@ class Literal(Token):
exc.pstr = instring exc.pstr = instring
raise exc raise exc
_L = Literal _L = Literal
ParserElement.literalStringClass = Literal
class Keyword(Token): class Keyword(Token):
"""Token to exactly match a specified string as a keyword, that is, it must be """Token to exactly match a specified string as a keyword, that is, it must be
immediately followed by a non-keyword character. Compare with C{Literal}:: immediately followed by a non-keyword character. Compare with C{L{Literal}}::
Literal("if") will match the leading C{'if'} in C{'ifAndOnlyIf'}. Literal("if") will match the leading C{'if'} in C{'ifAndOnlyIf'}.
Keyword("if") will not; it will only match the leading C{'if'} in C{'if x=1'}, or C{'if(y==2)'} Keyword("if") will not; it will only match the leading C{'if'} in C{'if x=1'}, or C{'if(y==2)'}
Accepts two optional constructor arguments in addition to the keyword string: Accepts two optional constructor arguments in addition to the keyword string:
@ -1821,9 +1795,9 @@ class QuotedString(Token):
- quoteChar - string of one or more characters defining the quote delimiting string - quoteChar - string of one or more characters defining the quote delimiting string
- escChar - character to escape quotes, typically backslash (default=None) - escChar - character to escape quotes, typically backslash (default=None)
- escQuote - special quote sequence to escape an embedded quote string (such as SQL's "" to escape an embedded ") (default=None) - escQuote - special quote sequence to escape an embedded quote string (such as SQL's "" to escape an embedded ") (default=None)
- multiline - boolean indicating whether quotes can span multiple lines (default=False) - multiline - boolean indicating whether quotes can span multiple lines (default=C{False})
- unquoteResults - boolean indicating whether the matched text should be unquoted (default=True) - unquoteResults - boolean indicating whether the matched text should be unquoted (default=C{True})
- endQuoteChar - string of one or more characters defining the end of the quote delimited string (default=None => same as quoteChar) - endQuoteChar - string of one or more characters defining the end of the quote delimited string (default=C{None} => same as quoteChar)
""" """
super(QuotedString,self).__init__() super(QuotedString,self).__init__()
@ -2003,7 +1977,7 @@ class White(Token):
by pyparsing grammars. This class is included when some whitespace structures by pyparsing grammars. This class is included when some whitespace structures
are significant. Define with a string containing the whitespace characters to be are significant. Define with a string containing the whitespace characters to be
matched; default is C{" \\t\\r\\n"}. Also takes optional C{min}, C{max}, and C{exact} arguments, matched; default is C{" \\t\\r\\n"}. Also takes optional C{min}, C{max}, and C{exact} arguments,
as defined for the C{Word} class.""" as defined for the C{L{Word}} class."""
whiteStrs = { whiteStrs = {
" " : "<SPC>", " " : "<SPC>",
"\t": "<TAB>", "\t": "<TAB>",
@ -2331,7 +2305,8 @@ class And(ParseExpression):
class _ErrorStop(Empty): class _ErrorStop(Empty):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super(Empty,self).__init__(*args, **kwargs) super(And._ErrorStop,self).__init__(*args, **kwargs)
self.name = '-'
self.leaveWhitespace() self.leaveWhitespace()
def __init__( self, exprs, savelist = True ): def __init__( self, exprs, savelist = True ):
@ -2359,8 +2334,7 @@ class And(ParseExpression):
loc, exprtokens = e._parse( instring, loc, doActions ) loc, exprtokens = e._parse( instring, loc, doActions )
except ParseSyntaxException: except ParseSyntaxException:
raise raise
except ParseBaseException: except ParseBaseException, pe:
pe = sys.exc_info()[1]
raise ParseSyntaxException(pe) raise ParseSyntaxException(pe)
except IndexError: except IndexError:
raise ParseSyntaxException( ParseException(instring, len(instring), self.errmsg, self) ) raise ParseSyntaxException( ParseException(instring, len(instring), self.errmsg, self) )
@ -2412,8 +2386,7 @@ class Or(ParseExpression):
for e in self.exprs: for e in self.exprs:
try: try:
loc2 = e.tryParse( instring, loc ) loc2 = e.tryParse( instring, loc )
except ParseException: except ParseException, err:
err = sys.exc_info()[1]
if err.loc > maxExcLoc: if err.loc > maxExcLoc:
maxException = err maxException = err
maxExcLoc = err.loc maxExcLoc = err.loc
@ -2436,7 +2409,7 @@ class Or(ParseExpression):
def __ixor__(self, other ): def __ixor__(self, other ):
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
return self.append( other ) #Or( [ self, other ] ) return self.append( other ) #Or( [ self, other ] )
def __str__( self ): def __str__( self ):
@ -2495,7 +2468,7 @@ class MatchFirst(ParseExpression):
def __ior__(self, other ): def __ior__(self, other ):
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal( other ) other = ParserElement.literalStringClass( other )
return self.append( other ) #MatchFirst( [ self, other ] ) return self.append( other ) #MatchFirst( [ self, other ] )
def __str__( self ): def __str__( self ):
@ -2916,13 +2889,14 @@ class Forward(ParseElementEnhance):
thereby leaving b and c out as parseable alternatives. It is recommended that you thereby leaving b and c out as parseable alternatives. It is recommended that you
explicitly group the values inserted into the C{Forward}:: explicitly group the values inserted into the C{Forward}::
fwdExpr << (a | b | c) fwdExpr << (a | b | c)
Converting to use the '<<=' operator instead will avoid this problem.
""" """
def __init__( self, other=None ): def __init__( self, other=None ):
super(Forward,self).__init__( other, savelist=False ) super(Forward,self).__init__( other, savelist=False )
def __lshift__( self, other ): def __lshift__( self, other ):
if isinstance( other, basestring ): if isinstance( other, basestring ):
other = Literal(other) other = ParserElement.literalStringClass(other)
self.expr = other self.expr = other
self.mayReturnEmpty = other.mayReturnEmpty self.mayReturnEmpty = other.mayReturnEmpty
self.strRepr = None self.strRepr = None
@ -2933,6 +2907,7 @@ class Forward(ParseElementEnhance):
self.saveAsList = self.expr.saveAsList self.saveAsList = self.expr.saveAsList
self.ignoreExprs.extend(self.expr.ignoreExprs) self.ignoreExprs.extend(self.expr.ignoreExprs)
return None return None
__ilshift__ = __lshift__
def leaveWhitespace( self ): def leaveWhitespace( self ):
self.skipWhitespace = False self.skipWhitespace = False
@ -2993,7 +2968,7 @@ class Upcase(TokenConverter):
DeprecationWarning,stacklevel=2) DeprecationWarning,stacklevel=2)
def postParse( self, instring, loc, tokenlist ): def postParse( self, instring, loc, tokenlist ):
return list(map( string.upper, tokenlist )) return list(map( str.upper, tokenlist ))
class Combine(TokenConverter): class Combine(TokenConverter):
@ -3029,7 +3004,7 @@ class Combine(TokenConverter):
return retToks return retToks
class Group(TokenConverter): class Group(TokenConverter):
"""Converter to return the matched tokens as a list - useful for returning tokens of C{ZeroOrMore} and C{OneOrMore} expressions.""" """Converter to return the matched tokens as a list - useful for returning tokens of C{L{ZeroOrMore}} and C{L{OneOrMore}} expressions."""
def __init__( self, expr ): def __init__( self, expr ):
super(Group,self).__init__( expr ) super(Group,self).__init__( expr )
self.saveAsList = True self.saveAsList = True
@ -3105,8 +3080,7 @@ def traceParseAction(f):
sys.stderr.write( ">>entering %s(line: '%s', %d, %s)\n" % (thisFunc,line(l,s),l,t) ) sys.stderr.write( ">>entering %s(line: '%s', %d, %s)\n" % (thisFunc,line(l,s),l,t) )
try: try:
ret = f(*paArgs) ret = f(*paArgs)
except Exception: except Exception, exc:
exc = sys.exc_info()[1]
sys.stderr.write( "<<leaving %s (exception: %s)\n" % (thisFunc,exc) ) sys.stderr.write( "<<leaving %s (exception: %s)\n" % (thisFunc,exc) )
raise raise
sys.stderr.write( "<<leaving %s (ret: %s)\n" % (thisFunc,ret) ) sys.stderr.write( "<<leaving %s (ret: %s)\n" % (thisFunc,ret) )
@ -3124,7 +3098,7 @@ def delimitedList( expr, delim=",", combine=False ):
"""Helper to define a delimited list of expressions - the delimiter defaults to ','. """Helper to define a delimited list of expressions - the delimiter defaults to ','.
By default, the list elements and delimiters can have intervening whitespace, and By default, the list elements and delimiters can have intervening whitespace, and
comments, but this can be overridden by passing C{combine=True} in the constructor. comments, but this can be overridden by passing C{combine=True} in the constructor.
If C{combine} is set to True, the matching tokens are returned as a single token If C{combine} is set to C{True}, the matching tokens are returned as a single token
string, with the delimiters included; otherwise, the matching tokens are returned string, with the delimiters included; otherwise, the matching tokens are returned
as a list of tokens, with the delimiters suppressed. as a list of tokens, with the delimiters suppressed.
""" """
@ -3226,7 +3200,7 @@ def _escapeRegexRangeChars(s):
def oneOf( strs, caseless=False, useRegex=True ): def oneOf( strs, caseless=False, useRegex=True ):
"""Helper to quickly define a set of alternative Literals, and makes sure to do """Helper to quickly define a set of alternative Literals, and makes sure to do
longest-first testing when there is a conflict, regardless of the input order, longest-first testing when there is a conflict, regardless of the input order,
but returns a C{MatchFirst} for best performance. but returns a C{L{MatchFirst}} for best performance.
Parameters: Parameters:
- strs - a string of space-delimited literals, or a list of string literals - strs - a string of space-delimited literals, or a list of string literals
@ -3284,7 +3258,7 @@ def oneOf( strs, caseless=False, useRegex=True ):
def dictOf( key, value ): def dictOf( key, value ):
"""Helper to easily and clearly define a dictionary by specifying the respective patterns """Helper to easily and clearly define a dictionary by specifying the respective patterns
for the key and value. Takes care of defining the C{Dict}, C{ZeroOrMore}, and C{Group} tokens for the key and value. Takes care of defining the C{L{Dict}}, C{L{ZeroOrMore}}, and C{L{Group}} tokens
in the proper order. The key pattern can include delimiting markers or punctuation, in the proper order. The key pattern can include delimiting markers or punctuation,
as long as they are suppressed, thereby leaving the significant key text. The value as long as they are suppressed, thereby leaving the significant key text. The value
pattern can include named results, so that the C{Dict} results can include named token pattern can include named results, so that the C{Dict} results can include named token
@ -3301,7 +3275,7 @@ def originalTextFor(expr, asString=True):
string containing the original parsed text. string containing the original parsed text.
If the optional C{asString} argument is passed as C{False}, then the return value is a If the optional C{asString} argument is passed as C{False}, then the return value is a
C{ParseResults} containing any results names that were originally matched, and a C{L{ParseResults}} containing any results names that were originally matched, and a
single token containing the original matched text from the input string. So if single token containing the original matched text from the input string. So if
the expression passed to C{L{originalTextFor}} contains expressions with defined the expression passed to C{L{originalTextFor}} contains expressions with defined
results names, you must set C{asString} to C{False} if you want to preserve those results names, you must set C{asString} to C{False} if you want to preserve those
@ -3335,7 +3309,7 @@ stringEnd = StringEnd().setName("stringEnd")
_escapedPunc = Word( _bslash, r"\[]-*.$+^?()~ ", exact=2 ).setParseAction(lambda s,l,t:t[0][1]) _escapedPunc = Word( _bslash, r"\[]-*.$+^?()~ ", exact=2 ).setParseAction(lambda s,l,t:t[0][1])
_printables_less_backslash = "".join([ c for c in printables if c not in r"\]" ]) _printables_less_backslash = "".join([ c for c in printables if c not in r"\]" ])
_escapedHexChar = Regex(r"\\0?[xX][0-9a-fA-F]+").setParseAction(lambda s,l,t:unichr(int(t[0][1:],16))) _escapedHexChar = Regex(r"\\0?[xX][0-9a-fA-F]+").setParseAction(lambda s,l,t:unichr(int(t[0].lstrip(r'\0x'),16)))
_escapedOctChar = Regex(r"\\0[0-7]+").setParseAction(lambda s,l,t:unichr(int(t[0][1:],8))) _escapedOctChar = Regex(r"\\0[0-7]+").setParseAction(lambda s,l,t:unichr(int(t[0][1:],8)))
_singleChar = _escapedPunc | _escapedHexChar | _escapedOctChar | Word(_printables_less_backslash,exact=1) _singleChar = _escapedPunc | _escapedHexChar | _escapedOctChar | Word(_printables_less_backslash,exact=1)
_charRange = Group(_singleChar + Suppress("-") + _singleChar) _charRange = Group(_singleChar + Suppress("-") + _singleChar)
@ -3376,7 +3350,7 @@ def matchOnlyAtCol(n):
def replaceWith(replStr): def replaceWith(replStr):
"""Helper method for common parse actions that simply return a literal value. Especially """Helper method for common parse actions that simply return a literal value. Especially
useful when used with C{transformString()}. useful when used with C{L{transformString<ParserElement.transformString>}()}.
""" """
def _replFunc(*args): def _replFunc(*args):
return [replStr] return [replStr]
@ -3398,7 +3372,7 @@ def downcaseTokens(s,l,t):
return [ tt.lower() for tt in map(_ustr,t) ] return [ tt.lower() for tt in map(_ustr,t) ]
def keepOriginalText(s,startLoc,t): def keepOriginalText(s,startLoc,t):
"""DEPRECATED - use new helper method C{originalTextFor}. """DEPRECATED - use new helper method C{L{originalTextFor}}.
Helper parse action to preserve original parsed text, Helper parse action to preserve original parsed text,
overriding any nested parse actions.""" overriding any nested parse actions."""
try: try:
@ -3464,7 +3438,7 @@ def makeXMLTags(tagStr):
def withAttribute(*args,**attrDict): def withAttribute(*args,**attrDict):
"""Helper to create a validating parse action to be used with start tags created """Helper to create a validating parse action to be used with start tags created
with C{makeXMLTags} or C{makeHTMLTags}. Use C{withAttribute} to qualify a starting tag with C{L{makeXMLTags}} or C{L{makeHTMLTags}}. Use C{withAttribute} to qualify a starting tag
with a required attribute value, to avoid false matches on common tags such as with a required attribute value, to avoid false matches on common tags such as
C{<TD>} or C{<DIV>}. C{<TD>} or C{<DIV>}.
@ -3499,7 +3473,7 @@ opAssoc = _Constants()
opAssoc.LEFT = object() opAssoc.LEFT = object()
opAssoc.RIGHT = object() opAssoc.RIGHT = object()
def operatorPrecedence( baseExpr, opList ): def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ):
"""Helper method for constructing grammars of expressions made up of """Helper method for constructing grammars of expressions made up of
operators working in a precedence hierarchy. Operators may be unary or operators working in a precedence hierarchy. Operators may be unary or
binary, left- or right-associative. Parse actions can also be attached binary, left- or right-associative. Parse actions can also be attached
@ -3518,13 +3492,15 @@ def operatorPrecedence( baseExpr, opList ):
be 1, 2, or 3) be 1, 2, or 3)
- rightLeftAssoc is the indicator whether the operator is - rightLeftAssoc is the indicator whether the operator is
right or left associative, using the pyparsing-defined right or left associative, using the pyparsing-defined
constants opAssoc.RIGHT and opAssoc.LEFT. constants C{opAssoc.RIGHT} and C{opAssoc.LEFT}.
- parseAction is the parse action to be associated with - parseAction is the parse action to be associated with
expressions matching this operator expression (the expressions matching this operator expression (the
parse action tuple member may be omitted) parse action tuple member may be omitted)
- lpar - expression for matching left-parentheses (default=Suppress('('))
- rpar - expression for matching right-parentheses (default=Suppress(')'))
""" """
ret = Forward() ret = Forward()
lastExpr = baseExpr | ( Suppress('(') + ret + Suppress(')') ) lastExpr = baseExpr | ( lpar + ret + rpar )
for i,operDef in enumerate(opList): for i,operDef in enumerate(opList):
opExpr,arity,rightLeftAssoc,pa = (operDef + (None,))[:4] opExpr,arity,rightLeftAssoc,pa = (operDef + (None,))[:4]
if arity == 3: if arity == 3:
@ -3569,6 +3545,7 @@ def operatorPrecedence( baseExpr, opList ):
lastExpr = thisExpr lastExpr = thisExpr
ret << lastExpr ret << lastExpr
return ret return ret
operatorPrecedence = infixNotation
dblQuotedString = Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\x[0-9a-fA-F]+)|(?:\\.))*"').setName("string enclosed in double quotes") dblQuotedString = Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\x[0-9a-fA-F]+)|(?:\\.))*"').setName("string enclosed in double quotes")
sglQuotedString = Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\x[0-9a-fA-F]+)|(?:\\.))*'").setName("string enclosed in single quotes") sglQuotedString = Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\x[0-9a-fA-F]+)|(?:\\.))*'").setName("string enclosed in single quotes")
@ -3715,8 +3692,7 @@ if __name__ == "__main__":
print ("tokens.columns = " + str(tokens.columns)) print ("tokens.columns = " + str(tokens.columns))
print ("tokens.tables = " + str(tokens.tables)) print ("tokens.tables = " + str(tokens.tables))
print (tokens.asXML("SQL",True)) print (tokens.asXML("SQL",True))
except ParseBaseException: except ParseBaseException, err:
err = sys.exc_info()[1]
print (teststring + "->") print (teststring + "->")
print (err.line) print (err.line)
print (" "*(err.column-1) + "^") print (" "*(err.column-1) + "^")

40
src/calibre/utils/speedup.c
View File

@ -3,6 +3,9 @@
#include <stdlib.h> #include <stdlib.h>
#define min(x, y) ((x < y) ? x : y)
#define max(x, y) ((x > y) ? x : y)
static PyObject * static PyObject *
speedup_parse_date(PyObject *self, PyObject *args) { speedup_parse_date(PyObject *self, PyObject *args) {
const char *raw, *orig, *tz; const char *raw, *orig, *tz;
@ -61,11 +64,48 @@ speedup_parse_date(PyObject *self, PyObject *args) {
(tzh*60 + tzm)*sign*60); (tzh*60 + tzm)*sign*60);
} }
static PyObject*
speedup_pdf_float(PyObject *self, PyObject *args) {
double f = 0.0, a = 0.0;
char *buf = "0", *dot;
void *free_buf = NULL;
int precision = 6, l = 0;
PyObject *ret;
if(!PyArg_ParseTuple(args, "d", &f)) return NULL;
a = fabs(f);
if (a > 1.0e-7) {
if(a > 1) precision = min(max(0, 6-(int)log10(a)), 6);
buf = PyOS_double_to_string(f, 'f', precision, 0, NULL);
if (buf != NULL) {
free_buf = (void*)buf;
if (precision > 0) {
l = strlen(buf) - 1;
while (l > 0 && buf[l] == '0') l--;
if (buf[l] == ',' || buf[l] == '.') buf[l] = 0;
else buf[l+1] = 0;
if ( (dot = strchr(buf, ',')) ) *dot = '.';
}
} else if (!PyErr_Occurred()) PyErr_SetString(PyExc_TypeError, "Float->str failed.");
}
ret = PyUnicode_FromString(buf);
if (free_buf != NULL) PyMem_Free(free_buf);
return ret;
}
static PyMethodDef speedup_methods[] = { static PyMethodDef speedup_methods[] = {
{"parse_date", speedup_parse_date, METH_VARARGS, {"parse_date", speedup_parse_date, METH_VARARGS,
"parse_date()\n\nParse ISO dates faster." "parse_date()\n\nParse ISO dates faster."
}, },
{"pdf_float", speedup_pdf_float, METH_VARARGS,
"pdf_float()\n\nConvert float to a string representation suitable for PDF"
},
{NULL, NULL, 0, NULL} {NULL, NULL, 0, NULL}
}; };

69
src/qt-harfbuzz/src/Makefile.am
View File

@ -1,69 +0,0 @@
## Process this file with automake to produce Makefile.in
noinst_LTLIBRARIES = libharfbuzz-1.la
MAINSOURCES = \
harfbuzz-buffer.c \
harfbuzz-stream.c \
harfbuzz-dump.c \
harfbuzz-gdef.c \
harfbuzz-gpos.c \
harfbuzz-gsub.c \
harfbuzz-impl.c \
harfbuzz-open.c \
harfbuzz-shaper.cpp \
harfbuzz-greek.c \
harfbuzz-tibetan.c \
harfbuzz-khmer.c \
harfbuzz-indic.cpp \
harfbuzz-hebrew.c \
harfbuzz-arabic.c \
harfbuzz-hangul.c \
harfbuzz-myanmar.c \
harfbuzz-thai.c
EXTRA_SOURCES = harfbuzz.c
PUBLICHEADERS = \
harfbuzz.h \
harfbuzz-buffer.h \
harfbuzz-dump.h \
harfbuzz-gdef.h \
harfbuzz-gpos.h \
harfbuzz-gsub.h \
harfbuzz-open.h \
harfbuzz-global.h \
harfbuzz-external.h \
harfbuzz-shaper.h \
harfbuzz-stream.h
PRIVATEHEADERS = \
harfbuzz-impl.h \
harfbuzz-buffer-private.h \
harfbuzz-stream-private.h \
harfbuzz-gdef-private.h \
harfbuzz-gpos-private.h \
harfbuzz-gsub-private.h \
harfbuzz-open-private.h \
harfbuzz-shaper-private.h
libharfbuzz_1_la_SOURCES = \
$(MAINSOURCES) \
$(PUBLICHEADERS) \
$(PRIVATEHEADERS)
#noinst_PROGRAMS = harfbuzz-dump
#
#harfbuzz_dump_SOURCES = \
# harfbuzz-dump-main.c
#
#harfbuzz_dump_LDADD = \
# libharfbuzz-1.la
EXTRA_DIST = \
README \
COPYING.FTL \
COPYING.GPL \
COPYING \
$(EXTRA_SOURCES)