Cutlery/calibre
1
0
Fork 0
mirror of https://github.com/kovidgoyal/calibre.git synced 2025-07-09 03:04:10 -04:00
Code Issues Packages Projects Releases Wiki Activity

Merge from trunk

Browse Source
This commit is contained in:
Charles Haley 2013-02-27 10:14:17 +01:00
commit fe9026c060
316 changed files with 134483 additions and 88556 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

206
Changelog.yaml
View File

@ -19,6 +19,212 @@
# new recipes:
# - title:
- version: 0.9.20
date: 2013-02-22
new features:
- title: "Book polishing: Add an option to smarten punctuation in the book when polishing"
- title: "Book polishing: Add an option to delete all saved settings to the load saved settings button"
- title: "Book polishing: Remember the last used settings"
- title: "Book polishing: Add a checkbox to enable/disable the detailed polishing report"
- title: "Add a separate tweak in Preferences-Tweaks for saving backups of files when polishing. That way you can have calibre save backups while converting EPUB->EPUB and not while polishing, if you so desire."
- title: "Content server: Allow clicking on the book cover to download it. Useful on small screen devices where clicking the Get button may be difficult"
- title: "Driver for Energy Systems C4 Touch."
tickets: [1127477]
bug fixes:
- title: "E-book viewer: Fix a bug that could cause the back button in the viewer to skip a location"
- title: "When tweaking/polishing an azw3 file that does not have an identified content ToC, do not auto-generate one."
tickets: [1130729]
- title: "Book polishing: Use the actual cover image dimensions when creating the svg wrapper for the cover image."
tickets: [1127273]
- title: "Book polishing: Do not error out on epub files containing an iTunesMetadata.plist file."
tickets: [1127308]
- title: "Book polishing: Fix trying to polish more than 5 books at a time not working"
- title: "Content server: Add workaround for bug in latest release of Google Chrome that causes it to not work with book lists containing some utf-8 characters"
tickets: [1130478]
- title: "E-book viewer: When viewing EPUB files, do not parse html as xhtml even if it has svg tags embedded. This allows malformed XHTML files to still be viewed."
- title: "Bulk metadata edit Search & recplace: Update the sample values when changing the type of identifier to search on"
- title: "Fix recipes with the / character in their names not useable from the command line"
tickets: [1127666]
- title: "News download: Fix regression that broke downloading of images in gif format"
- title: "EPUB/AZW3 Output: When splitting the output html on page breaks, handle page-break-after rules correctly, the pre split point html should contain the full element"
- title: "Fix stdout/stderr redirection temp files not being deleted when restarting calibre from within calibre on windows"
- title: "E-book viewer: When viewing epub files that have their cover marked as non-linear, show the cover at the start of the book instead of the end."
tickets: [1126030]
- title: "EPUB Input: Fix handling of cover references with fragments in the urls"
improved recipes:
- Fronda
- Various Polish news sources
new recipes:
- title: Pravda
author: Darko Miletic
- title: PNN
author: n.kucklaender
- title: Various Polish news sources
author: fenuks
- version: 0.9.19
date: 2013-02-15
new features:
- title: "New tool: \"Polish books\" that allows you to perform various automated cleanup actions on EPUB and AZW3 files without doing a full conversion."
type: major
description: "Polishing books is all about putting the shine of perfection on your ebook files. You can use it to subset embedded fonts, update the metadata in the book files from the metadata in the calibre library, manipulate the book jacket, etc. More features will be added in the future. To use this tool, go to Preferences->Toolbar and add the Polish books tool to the main toolbar. Then simply select the books you want to be polished and click the Polish books button. Polishing, unlike conversion, does not change the internal structure/markup of your book, it performs only the minimal set of actions needed to achieve its goals. Note that polish books is a completely new codebase, so there may well be bugs, polishing a book backs up the original as ORIGINAL_EPUB or ORIGINAL_AZW3, unless you have turned off this feature in Preferences->Tweaks, in which case you should backup your files manually. You can also use this tool from the command line with ebook-polish.exe."
- title: "Driver for the Trekstor Pyrus Mini."
tickets: [1124120]
- title: "E-book viewer: Add an option to change the minimum font size."
tickets: [1122333]
- title: "PDF Output: Add support for converting documents with math typesetting, as described here: http://manual.calibre-ebook.com/typesetting_math.html"
- title: "Column coloring/icons: Add more conditions when using date based columns with reference to 'today'."
bug fixes:
- title: "Transforming to titlecase - handle typographic hyphens in all caps phrases"
- title: "Dont ignore file open events that occur before the GUI is initialized on OS X"
tickets: [1122713]
- title: "News download: Handle feeds that have entries with empty ids"
- title: "Fix a regression that broke using the template editor"
- title: "Do not block startup while scanning the computer for available network interfaces. Speeds up startup time on some windows computers with lots of spurious network interfaces."
improved recipes:
- New Yorker
- Kommersant
- Le Monde (Subscription version)
- NZ Herald
new recipes:
- title: Navegalo
author: Douglas Delgado
- title: El Guardian and More Intelligent Life
author: Darko Miletic
- version: 0.9.18
date: 2013-02-08
new features:
- title: "New metadata source: Edelweiss, a catalog of books that is updated directly by publishers. To enable it, go to Preferences->Metadata download and enable the Edelweiss plugin."
tickets: [1091073]
- title: "Add an option to add extra spacing between rows in the book list. (Preferences->Look & Feel)"
tickets: [1117907]
- title: "Column coloring/icons: Add a 'days ago' condition, useable with columns that store dates to set colors/icons based on the number of days before today"
- title: "E-book viewer: Add shortcuts Ctrl+= and Ctrl+- to increase/decrease text size."
tickets: [ 1117524 ]
- title: "When showing possible duplicates after adding books, also show the file formats."
- title: "Driver for Trekstor Ventos Tablet"
bug fixes:
- title: "Conversion: When transliterating unicode characters, handle « and » correctly."
tickets: [1117270]
- title: "Fix adding books from multiple directories with multiple books per directory treating opf files as an ebook"
- title: "Fix download metadata window not resizable on smaller screens"
tickets: [1116849]
- title: "Tweak Book: When rebuilding azw3 files handle <a> tags that have name but not id attribute, these are apparently produced by kindlegen."
tickets: [ 1112934 ]
- title: "Fix regression in advanced column color rules."
tickets: [1118678]
improved recipes:
- El Mundo today
- fluter.de
- Birmingham Post
- Japan Times
- The Toronto Star
- Le Monde (subscription version)
- Globe and Mail
new recipes:
- title: VICE Magazine Deutschland
author: Alex
- title: Libertad Digital
author: Darko Miletic
- version: 0.9.17
date: 2013-02-01
new features:
- title: "Allow adding user specified icons to the main book list for books whose metadata matches specific criteria. Go to Preferences->Look & Feel->Column icons to setup these icons. They work in the same way as the column coloring rules."
type: major
- title: "Allow choosing which page of a PDF to use as the cover."
description: "To access this functionality add the PDF to calibre then click the edit metadata button. In the top right area of the edit metadata dialog there is a button to get the cover from the ebook file, this will now allow you to choose which page (from the first ten pages) of the pdf to use as the cover."
tickets: [1110019]
- title: "Add option to turn off reflections in the cover browser (Preferences->Look & Feel->Cover Browser)"
- title: "PDF Output: Add an option to add page numbers to the bottom of every page in the generated PDF file (look in the PDF Output section of the conversion dialog)"
- title: "Add the full item name to the tool tip of a leaf item displayed in the tag browser."
tickets: [1106231]
bug fixes:
- title: "Fix out-of-bounds data causing errors in the Tag Browser"
tickets: [1108017]
- title: "Conversion: Handle input documents that use multiple prefixes referring to the XHTML namespace correctly."
tickets: [1107220]
- title: "PDF Output: Fix regression that caused some svg images to be rendered as black rectangles."
tickets: [1105294]
- title: "Metadata download: Only normalize title case if the result has no language set or its language is English"
improved recipes:
- Baltimore Sun
- Harvard Business Review
- Victoria Times
- South China Morning Post
- Volksrant
- Seattle Times
new recipes:
- title: Dob NeviNosti
author: Darko Miletic
- title: La Nacion (CR)
author: Douglas Delgado
- version: 0.9.16
date: 2013-01-25

366
imgsrc/polish.svg Normal file
View File

@ -0,0 +1,366 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://web.resource.org/cc/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="48"
height="48"
id="svg2"
sodipodi:version="0.32"
inkscape:version="0.45"
version="1.0"
sodipodi:docname="edit-clear.svg"
inkscape:output_extension="org.inkscape.output.svg.inkscape"
sodipodi:docbase="/home/dobey/Projects/gnome-icon-theme/scalable/actions">
<defs
id="defs4">
<linearGradient
inkscape:collect="always"
id="linearGradient6019">
<stop
style="stop-color:#000000;stop-opacity:1;"
offset="0"
id="stop6021" />
<stop
style="stop-color:#000000;stop-opacity:0;"
offset="1"
id="stop6023" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5999">
<stop
style="stop-color:#c4a000;stop-opacity:1;"
offset="0"
id="stop6001" />
<stop
style="stop-color:#c4a000;stop-opacity:0;"
offset="1"
id="stop6003" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5987">
<stop
style="stop-color:#d7c20f;stop-opacity:1"
offset="0"
id="stop5989" />
<stop
style="stop-color:#b6970d;stop-opacity:1"
offset="1"
id="stop5991" />
</linearGradient>
<linearGradient
id="linearGradient5981"
inkscape:collect="always">
<stop
id="stop5983"
offset="0"
style="stop-color:#ffffff;stop-opacity:1;" />
<stop
id="stop5985"
offset="1"
style="stop-color:#ffffff;stop-opacity:0.69411765" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5996">
<stop
style="stop-color:#8f5902;stop-opacity:1;"
offset="0"
id="stop5998" />
<stop
style="stop-color:#73521e;stop-opacity:1"
offset="1"
id="stop6000" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5984">
<stop
style="stop-color:#ffffff;stop-opacity:1;"
offset="0"
id="stop5986" />
<stop
style="stop-color:#ffffff;stop-opacity:0.13438736"
offset="1"
id="stop5988" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5974">
<stop
style="stop-color:#ad7fa8;stop-opacity:1;"
offset="0"
id="stop5976" />
<stop
style="stop-color:#dac6d8;stop-opacity:1"
offset="1"
id="stop5978" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5966">
<stop
style="stop-color:#fdef72;stop-opacity:1"
offset="0"
id="stop5968" />
<stop
style="stop-color:#e2cb0b;stop-opacity:1"
offset="1"
id="stop5970" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient5958">
<stop
style="stop-color:#c17d11;stop-opacity:1;"
offset="0"
id="stop5960" />
<stop
style="stop-color:#e9b96e;stop-opacity:1"
offset="1"
id="stop5962" />
</linearGradient>
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5958"
id="linearGradient5964"
x1="28"
y1="16"
x2="26"
y2="8"
gradientUnits="userSpaceOnUse"
gradientTransform="translate(-2,0)" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5966"
id="linearGradient5972"
x1="20.933708"
y1="25.060659"
x2="30.208115"
y2="30.742676"
gradientUnits="userSpaceOnUse"
gradientTransform="translate(-2,0)" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5974"
id="linearGradient5980"
x1="27.651777"
y1="23.145937"
x2="21.59099"
y2="20.618719"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.9768193,0,0,1,-1.3746633,0)" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5984"
id="linearGradient5994"
gradientUnits="userSpaceOnUse"
x1="28"
y1="8"
x2="33.447109"
y2="16.685888" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5996"
id="linearGradient6002"
x1="30.324829"
y1="9.2407961"
x2="34"
y2="18"
gradientUnits="userSpaceOnUse"
gradientTransform="translate(-2,0)" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5981"
id="linearGradient5973"
gradientUnits="userSpaceOnUse"
x1="22.319767"
y1="41.955986"
x2="18.985712"
y2="37.029255"
gradientTransform="translate(-2,0)" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5987"
id="linearGradient5993"
x1="17.032078"
y1="27.446827"
x2="29.494455"
y2="37.845814"
gradientUnits="userSpaceOnUse"
gradientTransform="translate(-2,0)" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient5999"
id="linearGradient6005"
x1="27.354809"
y1="36.218422"
x2="23.489431"
y2="34.728424"
gradientUnits="userSpaceOnUse"
gradientTransform="translate(-2,0)" />
<radialGradient
inkscape:collect="always"
xlink:href="#linearGradient6019"
id="radialGradient6025"
cx="38"
cy="69"
fx="28.603323"
fy="69"
r="20"
gradientTransform="matrix(1,0,0,0.45,0,37.95)"
gradientUnits="userSpaceOnUse" />
</defs>
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#f6aaaa"
borderopacity="1"
inkscape:pageopacity="0.0"
inkscape:pageshadow="2"
inkscape:zoom="1"
inkscape:cx="37.14966"
inkscape:cy="21.336383"
inkscape:document-units="px"
inkscape:current-layer="layer1"
width="48px"
height="48px"
inkscape:showpageshadow="false"
showgrid="false"
gridspacingx="0.5px"
gridspacingy="0.5px"
gridempspacing="2"
inkscape:grid-points="true"
inkscape:window-width="862"
inkscape:window-height="875"
inkscape:window-x="12"
inkscape:window-y="50"
inkscape:object-paths="true"
inkscape:object-nodes="true"
objecttolerance="6"
gridtolerance="6"
guidetolerance="6"
showborder="false">
<inkscape:grid
type="xygrid"
id="grid5333"
spacingx="0.5px"
spacingy="0.5px"
empspacing="2" />
</sodipodi:namedview>
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:creator>
<cc:Agent>
<dc:title>Ulisse Perusin</dc:title>
</cc:Agent>
</dc:creator>
<cc:license
rdf:resource="http://creativecommons.org/licenses/GPL/2.0/" />
<dc:source>uli.peru@gmail.com</dc:source>
<dc:title>edit-clear</dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:label="Livello 1"
inkscape:groupmode="layer"
id="layer1">
<path
sodipodi:type="arc"
style="opacity:0.25;fill:url(#radialGradient6025);fill-opacity:1;stroke:none;stroke-width:0.99999994;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:20;stroke-dasharray:none;stroke-opacity:1"
id="path6017"
sodipodi:cx="38"
sodipodi:cy="69"
sodipodi:rx="20"
sodipodi:ry="9"
d="M 58,69 A 20,9 0 1 1 18,69 A 20,9 0 1 1 58,69 z"
transform="matrix(1,0,0,0.6666668,-13.999999,-5.0000087)" />
<path
style="fill:url(#linearGradient5964);fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient6002);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 34.59375,2.46875 C 26.390533,2.5744003 25.19695,18.716276 22.84375,21.625 L 26.84375,23.0625 C 29.475623,18.689953 42.599746,4.1545034 35.40625,2.5 C 35.12676,2.4690309 34.85837,2.4653419 34.59375,2.46875 z M 33.5625,4.53125 C 33.756063,4.5125114 33.930486,4.5369694 34.09375,4.625 C 34.746806,4.9771226 34.817405,6.1198771 34.25,7.15625 C 33.682595,8.1926229 32.684304,8.7583725 32.03125,8.40625 C 31.378197,8.0541272 31.307595,6.9113729 31.875,5.875 C 32.300554,5.0977202 32.981812,4.5874659 33.5625,4.53125 z"
id="path5371" />
<path
sodipodi:type="inkscape:offset"
inkscape:radius="-1.0049498"
inkscape:original="M 36.59375 2.46875 C 28.390533 2.5744003 27.19695 18.716276 24.84375 21.625 L 28.84375 23.0625 C 31.475623 18.689953 44.599746 4.1545034 37.40625 2.5 C 37.12676 2.4690309 36.85837 2.4653419 36.59375 2.46875 z "
style="opacity:0.26666667;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient5994);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
id="path5992"
d="M 36.59375,3.46875 C 34.872132,3.4909229 33.585825,4.3246243 32.40625,5.75 C 31.226675,7.1753757 30.257916,9.1916354 29.46875,11.34375 C 28.679584,13.495865 28.04471,15.77802 27.46875,17.71875 C 27.068859,19.066206 26.698893,20.125198 26.25,21.0625 L 28.4375,21.84375 C 30.056094,19.348126 33.476298,15.252572 35.96875,11.21875 C 37.294589,9.0729934 38.25245,7.0407089 38.46875,5.65625 C 38.5769,4.9640206 38.513818,4.4833206 38.34375,4.1875 C 38.179059,3.9010309 37.880274,3.6629145 37.21875,3.5 C 37.019255,3.4812162 36.817917,3.4658629 36.59375,3.46875 z"
transform="translate(-2,0)" />
<path
style="fill:url(#linearGradient5972);fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient5993);stroke-width:0.99999994;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:20;stroke-dasharray:none;stroke-opacity:1"
d="M 20.5,16.5 C 21.209506,18.503007 19.958612,20.237831 19.5,21.5 C 14.308433,23.045674 10.713199,31.203726 5.3674175,35.453585 C 6.0352055,36.150983 6.819644,36.897763 7.5,37.5 L 11.5625,33.96875 L 8.494944,38.493399 C 10.704181,40.284382 13,41.5 14.5,42 L 17.25,38.34375 L 15.5,42.5 C 16.951994,43.088882 20.485286,43.982025 22.5,44 L 24.50389,40.597503 L 23.990721,44.0625 C 24.820284,44.220859 26.428886,44.436716 27.5,44.46875 C 30.862186,38.96875 31.5,30 29.5,26 C 29,24 31,21.5 32.5,20.5 C 30,18.5 24.294411,16.196274 20.5,16.5 z"
id="path5367"
sodipodi:nodetypes="cccccccccccccccc" />
<path
style="opacity:0.26666667;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 9,38.5 C 13.816495,33.489105 13.465023,31.296074 19.116117,26.972272 C 16.133675,31.800703 15.650278,34.31233 12,40.5 L 9,38.5 z"
id="path5975"
sodipodi:nodetypes="cccc" />
<path
style="opacity:0.41568627;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 16.149808,42.202452 L 20.495835,32.362305 C 22.160348,29.378578 23.355507,26.392253 25.024808,24.014952 C 23.422854,29.432989 20.134118,36.136745 17.493558,42.639952 L 16.149808,42.202452 z"
id="path5979"
sodipodi:nodetypes="ccccc" />
<path
style="opacity:0.47843137;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient5973);stroke-width:0.99999994px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 21.625,17.5 C 21.721738,19.415146 20.702057,21.029645 20.40625,21.84375 C 20.302147,22.128459 20.07092,22.348124 19.78125,22.4375 C 14.327852,24.672144 12.151447,31.011538 6.7866116,35.523667 C 6.988254,35.723521 7.2786424,35.940971 7.4811412,36.131898 L 16.5,28.5 L 9.923385,38.310313 C 11.193418,39.337926 12.645586,40.194857 14.150041,40.799478 L 21.144394,31.5 L 16.869501,41.911612 C 18.46507,42.437269 19.967804,42.738908 21.81451,43 L 26.43324,35.3125 L 25.0625,43.219317 L 26.9375,43.445312 C 28.370713,40.909818 29.069882,37.778782 29.46875,34.65625 C 29.892695,31.337404 29.463786,28.115072 28.625,26.4375 C 28.597837,26.377291 28.576895,26.314465 28.5625,26.25 C 28.215642,24.862569 28.731642,23.504373 29.4375,22.375 C 29.864393,21.691971 30.367872,21.084221 30.902459,20.573223 C 29.730977,19.790532 28.315762,19.113157 26.53125,18.46875 C 24.769173,17.832444 23.033252,17.518725 21.625,17.5 z"
id="path6014"
sodipodi:nodetypes="csccccccccccccsssscsc" />
<path
style="opacity:0.24705882;fill:url(#linearGradient6005);fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 18.96875,43 C 21.146316,37.248129 25.364666,32.931057 26.985663,27.064588 C 27.037206,30.726661 27.235383,37.268314 25.09375,43.78125 C 24.773984,43.783025 24.919823,43.670441 24.62387,43.662697 L 25.424662,37.93818 L 22.143176,43.492564 C 19.952368,43.33624 20.848565,43.525163 18.96875,43 z"
id="path5977"
sodipodi:nodetypes="ccccccc" />
<path
style="opacity:0.48235294;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 21.003067,22.610447 C 19.751072,23.226826 18.940858,24.137725 18.019961,24.974835 C 19.246448,24.266192 20.398947,23.525841 22.019534,22.986097 L 21.003067,22.610447 z"
id="path5995"
sodipodi:nodetypes="cccc" />
<path
style="opacity:0.48235294;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 23.008698,23.061049 L 21.992233,25.049787 L 24.972946,23.461537 L 23.008698,23.061049 z"
id="path5997"
sodipodi:nodetypes="cccc" />
<path
style="opacity:0.48235294;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 22.939805,17.961399 L 22.044612,19.668421 L 23.610339,20.170505 L 22.939805,17.961399 z"
id="path6007"
sodipodi:nodetypes="cccc" />
<path
style="opacity:0.48235294;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 29.923254,19.88537 L 27.463006,21.720817 L 29.028733,22.222901 L 29.923254,19.88537 z"
id="path6009"
sodipodi:nodetypes="cccc" />
<path
style="opacity:0.48235294;fill:#c4a000;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M 26.012925,17.938566 L 24.499014,20.46318 L 26.064741,20.965264 L 26.012925,17.938566 z"
id="path6011"
sodipodi:nodetypes="cccc" />
<path
style="opacity:0.2;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1"
d="M 18.967726,22.024699 C 18.400946,19.059215 28.300561,24.177602 30.004548,25.019068 C 29.998066,26.098136 30.004548,27.019068 29.027729,27.019068 C 26.460191,25.626088 22.492474,23.413925 18.967726,22.024699 z"
id="path6013"
sodipodi:nodetypes="cccc" />
<path
style="fill:url(#linearGradient5980);fill-opacity:1;fill-rule:evenodd;stroke:#5c3566;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1"
d="M 18.650133,21.5 C 18.161723,20.5 18.650133,19.5 19.626953,19.5 C 23.618393,20.475417 26.951828,21.706232 30.371965,23.5 C 30.860375,24.5 30.371965,25.5 29.395146,25.5 C 25.861203,23.63558 22.528435,22.425187 18.650133,21.5 z"
id="path5373"
sodipodi:nodetypes="ccccc" />
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

20
manual/develop.rst
View File

@ -39,27 +39,27 @@ All the |app| python code is in the ``calibre`` package. This package contains t
* devices - All the device drivers. Just look through some of the built-in drivers to get an idea for how they work.
* For details, see: devices.interface which defines the interface supported by device drivers and devices.usbms which
* For details, see: devices.interface which defines the interface supported by device drivers and ``devices.usbms`` which
defines a generic driver that connects to a USBMS device. All USBMS based drivers in |app| inherit from it.
* ebooks - All the ebook conversion/metadata code. A good starting point is ``calibre.ebooks.conversion.cli`` which is the
module powering the :command:`ebook-convert` command. The conversion process is controlled via conversion.plumber.
The format independent code is all in ebooks.oeb and the format dependent code is in ebooks.format_name.
module powering the :command:`ebook-convert` command. The conversion process is controlled via ``conversion.plumber``.
The format independent code is all in ``ebooks.oeb`` and the format dependent code is in ``ebooks.format_name``.
* Metadata reading, writing, and downloading is all in ebooks.metadata
* Metadata reading, writing, and downloading is all in ``ebooks.metadata``
* Conversion happens in a pipeline, for the structure of the pipeline,
see :ref:`conversion-introduction`. The pipeline consists of an input
plugin, various transforms and an output plugin. The that code constructs
and drives the pipeline is in plumber.py. The pipeline works on a
and drives the pipeline is in :file:`plumber.py`. The pipeline works on a
representation of an ebook that is like an unzipped epub, with
manifest, spine, toc, guide, html content, etc. The
class that manages this representation is OEBBook in oeb/base.py. The
class that manages this representation is OEBBook in ``ebooks.oeb.base``. The
various transformations that are applied to the book during
conversions live in `oeb/transforms/*.py`. And the input and output
plugins live in `conversion/plugins/*.py`.
conversions live in :file:`oeb/transforms/*.py`. And the input and output
plugins live in :file:`conversion/plugins/*.py`.
* library - The database back-end and the content server. See library.database2 for the interface to the |app| library. library.server is the |app| Content Server.
* gui2 - The Graphical User Interface. GUI initialization happens in gui2.main and gui2.ui. The ebook-viewer is in gui2.viewer.
* library - The database back-end and the content server. See ``library.database2`` for the interface to the |app| library. ``library.server`` is the |app| Content Server.
* gui2 - The Graphical User Interface. GUI initialization happens in ``gui2.main`` and ``gui2.ui``. The ebook-viewer is in ``gui2.viewer``.
If you need help understanding the code, post in the `development forum <http://www.mobileread.com/forums/forumdisplay.php?f=240>`_
and you will most likely get help from one of |app|'s many developers.

110
manual/faq.rst
View File

@ -158,13 +158,23 @@ My device is not being detected by |app|?
Follow these steps to find the problem:
* Make sure that you are connecting only a single device to your computer at a time. Do not have another |app| supported device like an iPhone/iPad etc. at the same time.
* If you are connecting an Apple iDevice (iPad, iPod Touch, iPhone), use the 'Connect to iTunes' method in the 'Getting started' instructions in `Calibre + Apple iDevices: Start here <http://www.mobileread.com/forums/showthread.php?t=118559>`_.
* Make sure you are running the latest version of |app|. The latest version can always be downloaded from `the calibre website <http://calibre-ebook.com/download>`_.
* Ensure your operating system is seeing the device. That is, the device should show up in Windows Explorer (in Windows) or Finder (in OS X).
* Make sure that you are connecting only a single device to your computer
at a time. Do not have another |app| supported device like an iPhone/iPad
etc. at the same time.
* If you are connecting an Apple iDevice (iPad, iPod Touch, iPhone), use
the 'Connect to iTunes' method in the 'Getting started' instructions in
`Calibre + Apple iDevices: Start here <http://www.mobileread.com/forums/showthread.php?t=118559>`_.
* Make sure you are running the latest version of |app|. The latest version
can always be downloaded from `the calibre website <http://calibre-ebook.com/download>`_.
You can tell what version of |app| you are currently running by looking
at the bottom line of the main |app| window.
* Ensure your operating system is seeing the device. That is, the device
should show up in Windows Explorer (in Windows) or Finder (in OS X).
* In |app|, go to Preferences->Ignored Devices and check that your device
is not being ignored
* If all the above steps fail, go to Preferences->Miscellaneous and click debug device detection with your device attached and post the output as a ticket on `the calibre bug tracker <http://bugs.calibre-ebook.com>`_.
* If all the above steps fail, go to Preferences->Miscellaneous and click
debug device detection with your device attached and post the output as a
ticket on `the calibre bug tracker <http://bugs.calibre-ebook.com>`_.
My device is non-standard or unusual. What can I do to connect to it?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -240,42 +250,71 @@ If you don't want to uninstall it altogether, there are a couple of tricks you c
simplest is to simply re-name the executable file that launches the library program. More detail
`in the forums <http://www.mobileread.com/forums/showthread.php?t=65809>`_.
How do I use |app| with my iPad/iPhone/iTouch?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
How do I use |app| with my iPad/iPhone/iPod touch?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Over the air
^^^^^^^^^^^^^^
The easiest way to browse your |app| collection on your Apple device (iPad/iPhone/iPod) is by using the calibre content server, which makes your collection available over the net. First perform the following steps in |app|
The easiest way to browse your |app| collection on your Apple device
(iPad/iPhone/iPod) is by using the |app| content server, which makes your
collection available over the net. First perform the following steps in |app|
* Set the Preferred Output Format in |app| to EPUB (The output format can be set under :guilabel:`Preferences->Interface->Behavior`)
* Set the output profile to iPad (this will work for iPhone/iPods as well), under :guilabel:`Preferences->Conversion->Common Options->Page Setup`
* Convert the books you want to read on your iPhone to EPUB format by selecting them and clicking the Convert button.
* Turn on the Content Server in |app|'s preferences and leave |app| running.
* Set the Preferred Output Format in |app| to EPUB (The output format can be
set under :guilabel:`Preferences->Interface->Behavior`)
* Set the output profile to iPad (this will work for iPhone/iPods as well),
under :guilabel:`Preferences->Conversion->Common Options->Page Setup`
* Convert the books you want to read on your iDevice to EPUB format by
selecting them and clicking the Convert button.
* Turn on the Content Server by clicking the :guilabel:`Connect/Share` button
and leave |app| running. You can also tell |app| to automatically start the
content server via :guilabel:`Preferences->Sharing over the net`.
Now on your iPad/iPhone you have two choices, use either iBooks (version 1.2 and later) or Stanza (version 3.0 and later). Both are available free from the app store.
There are many apps for your iDevice that can connect to |app|. Here we
describe using two of them, iBooks and Stanza.
Using Stanza
***************
Now you should be able to access your books on your iPhone by opening Stanza. Go to "Get Books" and then click the "Shared" tab. Under Shared you will see an entry "Books in calibre". If you don't, make sure your iPad/iPhone is connected using the WiFi network in your house, not 3G. If the |app| catalog is still not detected in Stanza, you can add it manually in Stanza. To do this, click the "Shared" tab, then click the "Edit" button and then click "Add book source" to add a new book source. In the Add Book Source screen enter whatever name you like and in the URL field, enter the following::
You should be able to access your books on your iPhone by opening Stanza. Go to
"Get Books" and then click the "Shared" tab. Under Shared you will see an entry
"Books in calibre". If you don't, make sure your iPad/iPhone is connected using
the WiFi network in your house, not 3G. If the |app| catalog is still not
detected in Stanza, you can add it manually in Stanza. To do this, click the
"Shared" tab, then click the "Edit" button and then click "Add book source" to
add a new book source. In the Add Book Source screen enter whatever name you
like and in the URL field, enter the following::
http://192.168.1.2:8080/
Replace ``192.168.1.2`` with the local IP address of the computer running |app|. If you have changed the port the |app| content server is running on, you will have to change ``8080`` as well to the new port. The local IP address is the IP address you computer is assigned on your home network. A quick Google search will tell you how to find out your local IP address. Now click "Save" and you are done.
Replace ``192.168.1.2`` with the local IP address of the computer running
|app|. If you have changed the port the |app| content server is running on, you
will have to change ``8080`` as well to the new port. The local IP address is
the IP address you computer is assigned on your home network. A quick Google
search will tell you how to find out your local IP address. Now click "Save"
and you are done.
If you get timeout errors while browsing the calibre catalog in Stanza, try increasing the connection timeout value in the stanza settings. Go to Info->Settings and increase the value of Download Timeout.
If you get timeout errors while browsing the calibre catalog in Stanza, try
increasing the connection timeout value in the stanza settings. Go to
Info->Settings and increase the value of Download Timeout.
Using iBooks
**************
Start the Safari browser and type in the IP address and port of the computer running the calibre server, like this::
Start the Safari browser and type in the IP address and port of the computer
running the calibre server, like this::
http://192.168.1.2:8080/
Replace ``192.168.1.2`` with the local IP address of the computer running |app|. If you have changed the port the |app| content server is running on, you will have to change ``8080`` as well to the new port. The local IP address is the IP address you computer is assigned on your home network. A quick Google search will tell you how to find out your local IP address.
Replace ``192.168.1.2`` with the local IP address of the computer running
|app|. If you have changed the port the |app| content server is running on, you
will have to change ``8080`` as well to the new port. The local IP address is
the IP address you computer is assigned on your home network. A quick Google
search will tell you how to find out your local IP address.
You will see a list of books in Safari, just click on the epub link for whichever book you want to read, Safari will then prompt you to open it with iBooks.
You will see a list of books in Safari, just click on the epub link for
whichever book you want to read, Safari will then prompt you to open it with
iBooks.
With the USB cable + iTunes
@ -540,9 +579,9 @@ Yes, you can. Follow the instructions in the answer above for adding custom colu
How do I move my |app| library from one computer to another?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Simply copy the |app| library folder from the old to the new computer. You can find out what the library folder is by clicking the calibre icon in the toolbar. The very first item is the path to the library folder. Now on the new computer, start |app| for the first time. It will run the Welcome Wizard asking you for the location of the |app| library. Point it to the previously copied folder. If the computer you are transferring to already has a calibre installation, then the Welcome wizard wont run. In that case, click the calibre icon in the tooolbar and point it to the newly copied directory. You will now have two calibre libraries on your computer and you can switch between them by clicking the calibre icon on the toolbar.
Simply copy the |app| library folder from the old to the new computer. You can find out what the library folder is by clicking the calibre icon in the toolbar. The very first item is the path to the library folder. Now on the new computer, start |app| for the first time. It will run the Welcome Wizard asking you for the location of the |app| library. Point it to the previously copied folder. If the computer you are transferring to already has a calibre installation, then the Welcome wizard wont run. In that case, right-click the |app| icon in the tooolbar and point it to the newly copied directory. You will now have two calibre libraries on your computer and you can switch between them by clicking the |app| icon on the toolbar. Transferring your library in this manner preserver all your metadata, tags, custom columns, etc.
Note that if you are transferring between different types of computers (for example Windows to OS X) then after doing the above you should also right-click the calibre icon on the tool bar, select Library Maintenance and run the Check Library action. It will warn you about any problems in your library, which you should fix by hand.
Note that if you are transferring between different types of computers (for example Windows to OS X) then after doing the above you should also right-click the |app| icon on the tool bar, select Library Maintenance and run the Check Library action. It will warn you about any problems in your library, which you should fix by hand.
.. note:: A |app| library is just a folder which contains all the book files and their metadata. All the metadata is stored in a single file called metadata.db, in the top level folder. If this file gets corrupted, you may see an empty list of books in |app|. In this case you can ask |app| to restore your books by doing a right-click on the |app| icon in the toolbar and selecting Library Maintenance->Restore Library.
@ -577,7 +616,10 @@ or a Remote Desktop solution.
If you must share the actual library, use a file syncing tool like
DropBox or rsync or Microsoft SkyDrive instead of a networked drive. Even with
these tools there is danger of data corruption/loss, so only do this if you are
willing to live with that risk.
willing to live with that risk. In particular, be aware that **Google Drive**
is incompatible with |app|, if you put your |app| library in Google Drive, you
*will* suffer data loss. See
`this thread <http://www.mobileread.com/forums/showthread.php?t=205581>`_ for details.
Content From The Web
---------------------
@ -653,7 +695,7 @@ Post any output you see in a help message on the `Forum <http://www.mobileread.c
|app| freezes/crashes occasionally?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are three possible things I know of, that can cause this:
There are several possible things I know of, that can cause this:
* You recently connected an external monitor or TV to your computer. In
this case, whenever |app| opens a new window like the edit metadata
@ -661,10 +703,6 @@ There are three possible things I know of, that can cause this:
you dont notice it and so you think |app| has frozen. Disconnect your
second monitor and restart calibre.
* You are using a Wacom branded mouse. There is an incompatibility between
Wacom mice and the graphics toolkit |app| uses. Try using a non-Wacom
mouse.
* If you use RoboForm, it is known to cause |app| to crash. Add |app| to
the blacklist of programs inside RoboForm to fix this. Or uninstall
RoboForm.
@ -672,8 +710,22 @@ There are three possible things I know of, that can cause this:
* The Logitech SetPoint Settings application causes random crashes in
|app| when it is open. Close it before starting |app|.
* Constant Guard Protection by Xfinity causes crashes in |app|. You have to
manually allow |app| in it or uninstall Constant Guard Protection.
* Spybot - Search & Destroy blocks |app| from accessing its temporary files
breaking viewing and converting of books.
* You are using a Wacom branded USB mouse. There is an incompatibility between
Wacom mice and the graphics toolkit |app| uses. Try using a non-Wacom
mouse.
* On some 64 bit versions of Windows there are security software/settings
that prevent 64-bit |app| from working properly. If you are using the 64-bit
version of |app| try switching to the 32-bit version.
If none of the above apply to you, then there is some other program on your
computer that is interfering with |app|. First reboot your computer is safe
computer that is interfering with |app|. First reboot your computer in safe
mode, to have as few running programs as possible, and see if the crashes still
happen. If they do not, then you know it is some program causing the problem.
The most likely such culprit is a program that modifies other programs'
@ -784,7 +836,7 @@ Why doesn't |app| have an automatic update?
For many reasons:
* *There is no need to update every week*. If you are happy with how |app| works turn off the update notification and be on your merry way. Check back to see if you want to update once a year or so.
* Pre downloading the updates for all users in the background would mean require about 80TB of bandwidth *every week*. That costs thousands of dollars a month. And |app| is currently growing at 300,000 new users every month.
* Pre downloading the updates for all users in the background would require about 80TB of bandwidth *every week*. That costs thousands of dollars a month. And |app| is currently growing at 300,000 new users every month.
* If I implement a dialog that downloads the update and launches it, instead of going to the website as it does now, that would save the most ardent |app| updater, *at most five clicks a week*. There are far higher priority things to do in |app| development.
* If you really, really hate downloading |app| every week but still want to be up to the latest, I encourage you to run from source, which makes updating trivial. Instructions are :ref:`available here <develop>`.

2
manual/gui.rst
View File

@ -537,6 +537,8 @@ Calibre has several keyboard shortcuts to save you time and mouse movement. Thes
- Merge selected records, keeping originals
* - :kbd:`O`
- Open containing folder
* - :kbd:`P`
- Polish books
* - :kbd:`S`
- Save to Disk
* - :kbd:`V`

27
recipes/adventure_zone_pl.recipe
View File

@ -11,7 +11,7 @@ class Adventure_zone(BasicNewsRecipe):
max_articles_per_feed = 100
cover_url = 'http://www.adventure-zone.info/inne/logoaz_2012.png'
index='http://www.adventure-zone.info/fusion/'
use_embedded_content=False
use_embedded_content = False
preprocess_regexps = [(re.compile(r"<td class='capmain'>Komentarze</td>", re.IGNORECASE), lambda m: ''),
(re.compile(r'</?table.*?>'), lambda match: ''),
(re.compile(r'</?tbody.*?>'), lambda match: '')]
@ -21,7 +21,7 @@ class Adventure_zone(BasicNewsRecipe):
extra_css = '.main-bg{text-align: left;} td.capmain{ font-size: 22px; }'
feeds = [(u'Nowinki', u'http://www.adventure-zone.info/fusion/feeds/news.php')]
def parse_feeds (self):
'''def parse_feeds (self):
feeds = BasicNewsRecipe.parse_feeds(self)
soup=self.index_to_soup(u'http://www.adventure-zone.info/fusion/feeds/news.php')
tag=soup.find(name='channel')
@ -34,7 +34,7 @@ class Adventure_zone(BasicNewsRecipe):
for feed in feeds:
for article in feed.articles[:]:
article.title=titles[feed.articles.index(article)]
return feeds
return feeds'''
'''def get_cover_url(self):
@ -42,16 +42,25 @@ class Adventure_zone(BasicNewsRecipe):
cover=soup.find(id='box_OstatninumerAZ')
self.cover_url='http://www.adventure-zone.info/fusion/'+ cover.center.a.img['src']
return getattr(self, 'cover_url', self.cover_url)'''
def populate_article_metadata(self, article, soup, first):
result = re.search('(.+) - Adventure Zone', soup.title.string)
if result:
article.title = result.group(1)
else:
result = soup.body.find('strong')
if result:
article.title = result.string
def skip_ad_pages(self, soup):
skip_tag = soup.body.find(name='td', attrs={'class':'main-bg'})
skip_tag = skip_tag.findAll(name='a')
for r in skip_tag:
if r.strong:
word=r.strong.string.lower()
if word and (('zapowied' in word) or ('recenzj' in word) or ('solucj' in word) or ('poradnik' in word)):
return self.index_to_soup('http://www.adventure-zone.info/fusion/print.php?type=A&item'+r['href'][r['href'].find('article_id')+7:], raw=True)
title = soup.title.string.lower()
if (('zapowied' in title) or ('recenzj' in title) or ('solucj' in title) or ('poradnik' in title)):
for r in skip_tag:
if r.strong and r.strong.string:
word=r.strong.string.lower()
if (('zapowied' in word) or ('recenzj' in word) or ('solucj' in word) or ('poradnik' in word)):
return self.index_to_soup('http://www.adventure-zone.info/fusion/print.php?type=A&item'+r['href'][r['href'].find('article_id')+7:], raw=True)
def preprocess_html(self, soup):
footer=soup.find(attrs={'class':'news-footer middle-border'})

17
recipes/badania_net.recipe Normal file
View File

@ -0,0 +1,17 @@
from calibre.web.feeds.news import BasicNewsRecipe
class BadaniaNet(BasicNewsRecipe):
title = u'badania.net'
__author__ = 'fenuks'
description = u'chcesz wiedzieć więcej?'
category = 'science'
language = 'pl'
cover_url = 'http://badania.net/wp-content/badanianet_green_transparent.png'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
remove_empty_feeds = True
use_embedded_content = False
remove_tags = [dict(attrs={'class':['omc-flex-category', 'omc-comment-count', 'omc-single-tags']})]
remove_tags_after = dict(attrs={'class':'omc-single-tags'})
keep_only_tags = [dict(id='omc-full-article')]
feeds = [(u'Psychologia', u'http://badania.net/category/psychologia/feed/'), (u'Technologie', u'http://badania.net/category/technologie/feed/'), (u'Biologia', u'http://badania.net/category/biologia/feed/'), (u'Chemia', u'http://badania.net/category/chemia/feed/'), (u'Zdrowie', u'http://badania.net/category/zdrowie/'), (u'Seks', u'http://badania.net/category/psychologia-ewolucyjna-tematyka-seks/feed/')]

15
recipes/baltimore_sun.recipe
View File

@ -19,6 +19,7 @@ class BaltimoreSun(BasicNewsRecipe):
use_embedded_content = False
no_stylesheets = True
remove_javascript = True
#auto_cleanup = True
recursions = 1
ignore_duplicate_articles = {'title'}
@ -78,6 +79,7 @@ class BaltimoreSun(BasicNewsRecipe):
#(u'High School', u'http://www.baltimoresun.com/sports/high-school/rss2.0.xml'),
#(u'Outdoors', u'http://www.baltimoresun.com/sports/outdoors/rss2.0.xml'),
## Entertainment ##
(u'Celebrity News', u'http://www.baltimoresun.com/entertainment/celebrities/rss2.0.xml'),
(u'Arts & Theater', u'http://www.baltimoresun.com/entertainment/arts/rss2.0.xml'),
@ -142,12 +144,12 @@ class BaltimoreSun(BasicNewsRecipe):
(u'Read Street', u'http://www.baltimoresun.com/features/books/read-street/rss2.0.xml'),
(u'Z on TV', u'http://www.baltimoresun.com/entertainment/tv/z-on-tv-blog/rss2.0.xml'),
## Life Blogs ##
(u'BMore Green', u'http://weblogs.baltimoresun.com/features/green/index.xml'),
(u'Baltimore Insider',u'http://www.baltimoresun.com/features/baltimore-insider-blog/rss2.0.xml'),
(u'Homefront', u'http://www.baltimoresun.com/features/parenting/homefront/rss2.0.xml'),
(u'Picture of Health', u'http://www.baltimoresun.com/health/blog/rss2.0.xml'),
(u'Unleashed', u'http://weblogs.baltimoresun.com/features/mutts/blog/index.xml'),
### Life Blogs ##
#(u'BMore Green', u'http://weblogs.baltimoresun.com/features/green/index.xml'),
#(u'Baltimore Insider',u'http://www.baltimoresun.com/features/baltimore-insider-blog/rss2.0.xml'),
#(u'Homefront', u'http://www.baltimoresun.com/features/parenting/homefront/rss2.0.xml'),
#(u'Picture of Health', u'http://www.baltimoresun.com/health/blog/rss2.0.xml'),
#(u'Unleashed', u'http://weblogs.baltimoresun.com/features/mutts/blog/index.xml'),
## b the site blogs ##
(u'Game Cache', u'http://www.baltimoresun.com/entertainment/bthesite/game-cache/rss2.0.xml'),
@ -167,6 +169,7 @@ class BaltimoreSun(BasicNewsRecipe):
]
def get_article_url(self, article):
ans = None
try:

6
recipes/bash_org_pl.recipe
View File

@ -35,8 +35,8 @@ class Bash_org_pl(BasicNewsRecipe):
soup=self.index_to_soup(u'http://bash.org.pl/random/')
#date=soup.find('div', attrs={'class':'right'}).string
url=soup.find('a', attrs={'class':'qid click'})
title=url.string
url='http://bash.org.pl' +url['href']
title=''
url='http://bash.org.pl/random/'
articles.append({'title' : title,
'url' : url,
'date' : '',
@ -44,6 +44,8 @@ class Bash_org_pl(BasicNewsRecipe):
})
return articles
def populate_article_metadata(self, article, soup, first):
article.title = soup.find(attrs={'class':'qid click'}).string
def parse_index(self):
feeds = []

23
recipes/birmingham_post.recipe
View File

@ -14,7 +14,7 @@ class AdvancedUserRecipe1306097511(BasicNewsRecipe):
remove_empty_feeds = True
remove_javascript = True
no_stylesheets = True
#auto_cleanup = True
auto_cleanup = True
language = 'en_GB'
cover_url = 'http://profile.ak.fbcdn.net/hprofile-ak-snc4/161987_9010212100_2035706408_n.jpg'
@ -23,7 +23,7 @@ class AdvancedUserRecipe1306097511(BasicNewsRecipe):
def get_cover_url(self):
soup = self.index_to_soup('http://www.birminghampost.net')
# look for the block containing the sun button and url
cov = soup.find(attrs={'height' : re.compile('3'), 'alt' : re.compile('Birmingham Post')})
cov = soup.find(attrs={'height' : re.compile('3'), 'alt' : re.compile('Post')})
print
print '%%%%%%%%%%%%%%%',cov
print
@ -43,20 +43,7 @@ class AdvancedUserRecipe1306097511(BasicNewsRecipe):
return cover_url
keep_only_tags = [
dict(attrs={'id' : 'article-header'}),
#dict(name='h1',attrs={'id' : 'article-header'}),
dict(attrs={'class':['article-meta-author','article-meta-date','article main','art-o art-align-center otm-1 ']}),
dict(name='div',attrs={'class' : 'article-image full'}),
dict(attrs={'clas' : 'art-o art-align-center otm-1 '}),
dict(name='div',attrs={'class' : 'article main'}),
#dict(name='p')
#dict(attrs={'id' : 'three-col'})
]
remove_tags = [
# dict(name='div',attrs={'class' : 'span-33 last header-links'})
]
feeds = [
#(u'News',u'http://www.birminghampost.net/news/rss.xml'),
(u'West Mids. News', u'http://www.birminghampost.net/news/west-midlands-news/rss.xml'),
@ -65,9 +52,3 @@ class AdvancedUserRecipe1306097511(BasicNewsRecipe):
(u'Bloggs & Comments',u'http://www.birminghampost.net/comment/rss.xml')
]
extra_css = '''
h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;text-align:center;}
h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
'''

27
recipes/democracy_journal.recipe Normal file
View File

@ -0,0 +1,27 @@
from calibre.web.feeds.news import BasicNewsRecipe
import re
class AdvancedUserRecipe1361743898(BasicNewsRecipe):
title = u'Democracy Journal'
description = '''A journal of ideas. Published quarterly.'''
__author__ = u'David Nye'
language = 'en'
oldest_article = 90
max_articles_per_feed = 30
no_stylesheets = True
auto_cleanup = True
def parse_index(self):
articles = []
feeds = []
soup = self.index_to_soup("http://www.democracyjournal.org")
for x in soup.findAll(href=re.compile("http://www\.democracyjournal\.org/\d*/.*php$")):
url = x.get('href')
title = self.tag_to_string(x)
articles.append({'title':title, 'url':url, 'description':'', 'date':''})
feeds.append(('Articles', articles))
return feeds
def print_version(self, url):
return url + '?page=all'

15
recipes/discover_magazine.recipe
View File

@ -33,6 +33,21 @@ class DiscoverMagazine(BasicNewsRecipe):
remove_tags_after = [dict(name='div', attrs={'class':'listingBar'})]
# Login stuff
needs_subscription = True
use_javascript_to_login = True
requires_version = (0, 9, 20)
def javascript_login(self, br, username, password):
br.visit('http://discovermagazine.com', timeout=120)
f = br.select_form('div.login.section div.form')
f['username'] = username
f['password'] = password
br.submit('input[id="signInButton"]', timeout=120)
br.run_for_a_time(20)
# End login stuff
def append_page(self, soup, appendtag, position):
pager = soup.find('span',attrs={'class':'next'})
if pager:

46
recipes/dobanevinosti.recipe Normal file
View File

@ -0,0 +1,46 @@
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
dobanevinosti.blogspot.com
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe
class DobaNevinosti(BasicNewsRecipe):
title = 'Doba Nevinosti'
__author__ = 'Darko Miletic'
description = 'Filmski blog'
oldest_article = 15
max_articles_per_feed = 100
language = 'sr'
encoding = 'utf-8'
no_stylesheets = True
use_embedded_content = True
publication_type = 'blog'
auto_cleanup = True
extra_css = """
@font-face {font-family: "sans1";src:url(res:///opt/sony/ebook/FONT/tt0003m_.ttf)}
body{font-family: "Trebuchet MS",Trebuchet,Verdana,sans1,sans-serif}
img{margin-bottom: 0.8em; display:block;}
"""
conversion_options = {
'comment' : description
, 'tags' : 'film, blog, srbija, tv'
, 'publisher': 'Dimitrije Vojinov'
, 'language' : language
}
remove_attributes = ['lang', 'border']
preprocess_regexps = [(re.compile(u'\u0110'), lambda match: u'\u00D0')]
feeds = [(u'Tekstovi', u'http://dobanevinosti.blogspot.com/feeds/posts/default')]
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll('img'):
if not item.has_key('alt'):
item['alt'] = 'image'
return soup

3
recipes/ekologia_pl.recipe
View File

@ -15,7 +15,8 @@ class EkologiaPl(BasicNewsRecipe):
no_stylesheets = True
remove_empty_feeds = True
use_embedded_content = False
remove_tags = [dict(attrs={'class':['ekoLogo', 'powrocArt', 'butonDrukuj']})]
remove_attrs = ['style']
remove_tags = [dict(attrs={'class':['ekoLogo', 'powrocArt', 'butonDrukuj', 'widget-social-buttons']})]
feeds = [(u'Wiadomo\u015bci', u'http://www.ekologia.pl/rss/20,53,0'), (u'\u015arodowisko', u'http://www.ekologia.pl/rss/20,56,0'), (u'Styl \u017cycia', u'http://www.ekologia.pl/rss/20,55,0')]

27
recipes/el_malpensante.recipe Normal file
View File

@ -0,0 +1,27 @@
# coding=utf-8
# https://github.com/iemejia/calibrecolombia
'''
http://www.elmalpensante.com/
'''
from calibre.web.feeds.news import BasicNewsRecipe
class ElMalpensante(BasicNewsRecipe):
title = u'El Malpensante'
language = 'es_CO'
__author__ = 'Ismael Mejia <iemejia@gmail.com>'
cover_url = 'http://elmalpensante.com/img/layout/logo.gif'
description = 'El Malpensante'
oldest_article = 30
simultaneous_downloads = 20
#tags = 'news, sport, blog'
use_embedded_content = True
remove_empty_feeds = True
max_articles_per_feed = 100
feeds = [(u'Artículos', u'http://www.elmalpensante.com/articulosRSS.php'),
(u'Malpensantías', u'http://www.elmalpensante.com/malpensantiasRSS.php'),
(u'Margaritas', u'http://www.elmalpensante.com/margaritasRSS.php'),
# This one is almost the same as articulos so we leave articles
# (u'Noticias', u'http://www.elmalpensante.com/noticiasRSS.php'),
]

28
recipes/el_mundo_today.recipe
View File

@ -3,29 +3,34 @@ from calibre.web.feeds.news import BasicNewsRecipe
class ElMundoTodayRecipe(BasicNewsRecipe):
title = 'El Mundo Today'
__author__ = 'atordo'
description = u'La actualidad del mañana'
description = u'La actualidad del ma\u00f1ana'
category = 'Noticias, humor'
cover_url = 'http://www.elmundotoday.com/wp-content/themes/EarthlyTouch/images/logo.png'
oldest_article = 30
oldest_article = 15
max_articles_per_feed = 60
auto_cleanup = False
no_stylesheets = True
remove_javascript = True
language = 'es'
use_embedded_content = False
publication_type = 'blog'
preprocess_regexps = [
(re.compile(r'</title>.*<!--Begin Article Single-->', re.DOTALL),
lambda match: '</title><body>'),
#(re.compile(r'^\t{5}<a href.*Permanent Link to ">$'), lambda match: ''),
#(re.compile(r'\t{5}</a>$'), lambda match: ''),
(re.compile(r'<div class="social4i".*</body>', re.DOTALL),
lambda match: '</body>'),
lambda match: '</title></head><body>'),
(re.compile(r'<img alt="" src="http://www.elmundotoday.com/wp-content/themes/emt/images/otrassecciones-line.gif">'),
lambda match: ''),
(re.compile(r'<div style="clear: both;"></div>.*</body>', re.DOTALL),
lambda match: '</body>')
]
keep_only_tags = [
dict(name='div', attrs={'class':'post-wrapper'})
dict(name='div', attrs={'class':'post-wrapper '})
]
remove_tags = [
dict(name='div', attrs={'class':'social4i'}),
dict(name='span', attrs={'class':'num-comentarios'})
]
remove_attributes = [ 'href', 'title', 'alt' ]
@ -36,8 +41,3 @@ class ElMundoTodayRecipe(BasicNewsRecipe):
'''
feeds = [('El Mundo Today', 'http://www.elmundotoday.com/feed/')]
def get_broser(self):
br = BasicNewsRecipe.get_browser(self)
br.set_handle_gzip(True)
return br

93
recipes/elguardian.recipe Normal file
View File

@ -0,0 +1,93 @@
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
elguardian.com.ar
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe
class ElGuardian(BasicNewsRecipe):
title = 'El Guardian'
__author__ = 'Darko Miletic'
description = "Semanario con todas las tendencias de un pais"
publisher = 'Editorial Apache SA'
category = 'news,politics,Argentina'
oldest_article = 8
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'utf8'
use_embedded_content = False
language = 'es_AR'
remove_empty_feeds = True
publication_type = 'magazine'
issn = '1666-7476'
masthead_url = 'http://elguardian.com.ar/application/templates/frontend/images/home/logo.png'
extra_css = """
body{font-family: Arial,sans-serif}
img{margin-bottom: 0.4em; display:block}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
, 'series' : title
, 'isbn' : issn
}
keep_only_tags = [dict(attrs={'class':['fotos', 'header_nota', 'nota']})]
remove_tags = [dict(name=['meta','link','iframe','embed','object'])]
remove_attributes = ['lang']
feeds = [
(u'El Pais' , u'http://elguardian.com.ar/RSS/el-pais.xml' )
,(u'Columnistas' , u'http://elguardian.com.ar/RSS/columnistas.xml' )
,(u'Personajes' , u'http://elguardian.com.ar/RSS/personajes.xml' )
,(u'Tinta roja' , u'http://elguardian.com.ar/RSS/tinta-roja.xml' )
,(u'Yo fui' , u'http://elguardian.com.ar/RSS/yo-fui.xml' )
,(u'Ciencia' , u'http://elguardian.com.ar/RSS/ciencia.xml' )
,(u'Cronicas' , u'http://elguardian.com.ar/RSS/cronicas.xml' )
,(u'Culturas' , u'http://elguardian.com.ar/RSS/culturas.xml' )
,(u'DxT' , u'http://elguardian.com.ar/RSS/dxt.xml' )
,(u'Fierros' , u'http://elguardian.com.ar/RSS/fierros.xml' )
,(u'Frente fashion', u'http://elguardian.com.ar/RSS/frente-fashion.xml')
,(u'Pan y vino' , u'http://elguardian.com.ar/RSS/pan-y-vino.xml' )
,(u'Turismo' , u'http://elguardian.com.ar/RSS/turismo.xml' )
]
def get_cover_url(self):
soup = self.index_to_soup('http://elguardian.com.ar/')
udata = soup.find('div', attrs={'class':'datosNumero'})
if udata:
sdata = udata.find('div')
if sdata:
stra = re.findall(r'\d+', self.tag_to_string(sdata))
self.conversion_options.update({'series_index':int(stra[1])})
unumero = soup.find('div', attrs={'class':'ultimoNumero'})
if unumero:
img = unumero.find('img', src=True)
if img:
return img['src']
return None
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll('a'):
limg = item.find('img')
if item.string is not None:
str = item.string
item.replaceWith(str)
else:
if limg:
item.name = 'div'
item.attrs = []
else:
str = self.tag_to_string(item)
item.replaceWith(str)
for item in soup.findAll('img'):
if not item.has_key('alt'):
item['alt'] = 'image'
return soup

23
recipes/eso_pl.recipe Normal file
View File

@ -0,0 +1,23 @@
from calibre.web.feeds.news import BasicNewsRecipe
class ESO(BasicNewsRecipe):
title = u'ESO PL'
__author__ = 'fenuks'
description = u'ESO, Europejskie Obserwatorium Południowe, buduje i obsługuje najbardziej zaawansowane naziemne teleskopy astronomiczne na świecie'
category = 'astronomy'
language = 'pl'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
remove_empty_feeds = True
use_embedded_content = False
cover_url = 'https://twimg0-a.akamaihd.net/profile_images/1922519424/eso-twitter-logo.png'
keep_only_tags = [dict(attrs={'class':'subcl'})]
remove_tags = [dict(id='lang_row'), dict(attrs={'class':['pr_typeid', 'pr_news_feature_link', 'outreach_usage', 'hidden']})]
feeds = [(u'Wiadomo\u015bci', u'http://www.eso.org/public/poland/news/feed/'), (u'Og\u0142oszenia', u'http://www.eso.org/public/poland/announcements/feed/'), (u'Zdj\u0119cie tygodnia', u'http://www.eso.org/public/poland/images/potw/feed/')]
def preprocess_html(self, soup):
for a in soup.findAll('a', href=True):
if a['href'].startswith('/'):
a['href'] = 'http://www.eso.org' + a['href']
return soup

182
recipes/financial_times_us.recipe Normal file
View File

@ -0,0 +1,182 @@
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
http://www.ft.com/intl/us-edition
'''
import datetime
from calibre.ptempfile import PersistentTemporaryFile
from calibre import strftime
from calibre.web.feeds.news import BasicNewsRecipe
class FinancialTimes(BasicNewsRecipe):
title = 'Financial Times (US) printed edition'
__author__ = 'Darko Miletic'
description = "The Financial Times (FT) is one of the world's leading business news and information organisations, recognised internationally for its authority, integrity and accuracy."
publisher = 'The Financial Times Ltd.'
category = 'news, finances, politics, UK, World'
oldest_article = 2
language = 'en'
max_articles_per_feed = 250
no_stylesheets = True
use_embedded_content = False
needs_subscription = True
encoding = 'utf8'
publication_type = 'newspaper'
articles_are_obfuscated = True
temp_files = []
masthead_url = 'http://im.media.ft.com/m/img/masthead_main.jpg'
LOGIN = 'https://registration.ft.com/registration/barrier/login'
LOGIN2 = 'http://media.ft.com/h/subs3.html'
INDEX = 'http://www.ft.com/intl/us-edition'
PREFIX = 'http://www.ft.com'
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
, 'linearize_tables' : True
}
def get_browser(self):
br = BasicNewsRecipe.get_browser(self)
br.open(self.INDEX)
if self.username is not None and self.password is not None:
br.open(self.LOGIN2)
br.select_form(name='loginForm')
br['username'] = self.username
br['password'] = self.password
br.submit()
return br
keep_only_tags = [
dict(name='div' , attrs={'class':['fullstory fullstoryHeader', 'ft-story-header']})
,dict(name='div' , attrs={'class':'standfirst'})
,dict(name='div' , attrs={'id' :'storyContent'})
,dict(name='div' , attrs={'class':['ft-story-body','index-detail']})
,dict(name='h2' , attrs={'class':'entry-title'} )
,dict(name='span', attrs={'class':lambda x: x and 'posted-on' in x.split()} )
,dict(name='span', attrs={'class':'author_byline'} )
,dict(name='div' , attrs={'class':'entry-content'} )
]
remove_tags = [
dict(name='div', attrs={'id':'floating-con'})
,dict(name=['meta','iframe','base','object','embed','link'])
,dict(attrs={'class':['storyTools','story-package','screen-copy','story-package separator','expandable-image']})
]
remove_attributes = ['width','height','lang']
extra_css = """
body{font-family: Georgia,Times,"Times New Roman",serif}
h2{font-size:large}
.ft-story-header{font-size: x-small}
.container{font-size:x-small;}
h3{font-size:x-small;color:#003399;}
.copyright{font-size: x-small}
img{margin-top: 0.8em; display: block}
.lastUpdated{font-family: Arial,Helvetica,sans-serif; font-size: x-small}
.byline,.ft-story-body,.ft-story-header{font-family: Arial,Helvetica,sans-serif}
"""
def get_artlinks(self, elem):
articles = []
count = 0
for item in elem.findAll('a',href=True):
count = count + 1
if self.test and count > 2:
return articles
rawlink = item['href']
url = rawlink
if not rawlink.startswith('http://'):
url = self.PREFIX + rawlink
try:
urlverified = self.browser.open_novisit(url).geturl() # resolve redirect.
except:
continue
title = self.tag_to_string(item)
date = strftime(self.timefmt)
articles.append({
'title' :title
,'date' :date
,'url' :urlverified
,'description':''
})
return articles
def parse_index(self):
feeds = []
soup = self.index_to_soup(self.INDEX)
dates= self.tag_to_string(soup.find('div', attrs={'class':'btm-links'}).find('div'))
self.timefmt = ' [%s]'%dates
wide = soup.find('div',attrs={'class':'wide'})
if not wide:
return feeds
allsections = wide.findAll(attrs={'class':lambda x: x and 'footwell' in x.split()})
if not allsections:
return feeds
count = 0
for item in allsections:
count = count + 1
if self.test and count > 2:
return feeds
fitem = item.h3
if not fitem:
fitem = item.h4
ftitle = self.tag_to_string(fitem)
self.report_progress(0, _('Fetching feed')+' %s...'%(ftitle))
feedarts = self.get_artlinks(item.ul)
feeds.append((ftitle,feedarts))
return feeds
def preprocess_html(self, soup):
items = ['promo-box','promo-title',
'promo-headline','promo-image',
'promo-intro','promo-link','subhead']
for item in items:
for it in soup.findAll(item):
it.name = 'div'
it.attrs = []
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll('a'):
limg = item.find('img')
if item.string is not None:
str = item.string
item.replaceWith(str)
else:
if limg:
item.name = 'div'
item.attrs = []
else:
str = self.tag_to_string(item)
item.replaceWith(str)
for item in soup.findAll('img'):
if not item.has_key('alt'):
item['alt'] = 'image'
return soup
def get_cover_url(self):
cdate = datetime.date.today()
if cdate.isoweekday() == 7:
cdate -= datetime.timedelta(days=1)
return cdate.strftime('http://specials.ft.com/vtf_pdf/%d%m%y_FRONT1_USA.pdf')
def get_obfuscated_article(self, url):
count = 0
while (count < 10):
try:
response = self.browser.open(url)
html = response.read()
count = 10
except:
print "Retrying download..."
count += 1
tfile = PersistentTemporaryFile('_fa.html')
tfile.write(html)
tfile.close()
self.temp_files.append(tfile)
return tfile.name
def cleanup(self):
self.browser.open('https://registration.ft.com/registration/login/logout?location=')

17
recipes/fluter_de.recipe
View File

@ -14,26 +14,17 @@ class AdvancedUserRecipe1313693926(BasicNewsRecipe):
language = 'de'
encoding = 'UTF-8'
__author__ = 'Armin Geller' # 2011-08-19
__author__ = 'Armin Geller' # 2013-02-05 V3
oldest_article = 7
max_articles_per_feed = 50
remove_tags = [
dict(name='div', attrs={'id':["comments"]}),
dict(attrs={'class':['commentlink']}),
]
keep_only_tags = [
dict(name='div', attrs={'class':["grid_8 articleText"]}),
dict(name='div', attrs={'class':["articleTextInnerText"]}),
]
feeds = [
(u'Inhalt:', u'http://www.fluter.de/de/?tpl=907'),
]
extra_css = '.cs_img {margin-right: 10pt;}'
def print_version(self, url):
return url + '?tpl=1260'

5
recipes/fronda.recipe
View File

@ -23,7 +23,6 @@ class Fronda(BasicNewsRecipe):
extra_css = '''
h1 {font-size:150%}
.body {text-align:left;}
div.headline {font-weight:bold}
'''
earliest_date = date.today() - timedelta(days=oldest_article)
@ -72,7 +71,7 @@ class Fronda(BasicNewsRecipe):
feeds.append((genName, articles[genName]))
return feeds
keep_only_tags = [
keep_only_tags = [
dict(name='div', attrs={'class':'yui-g'})
]
@ -84,5 +83,7 @@ class Fronda(BasicNewsRecipe):
dict(name='ul', attrs={'class':'comment-list'}),
dict(name='ul', attrs={'class':'category'}),
dict(name='p', attrs={'id':'comments-disclaimer'}),
dict(name='div', attrs={'style':'text-align: left; margin-bottom: 15px;'}),
dict(name='div', attrs={'style':'text-align: left; margin-top: 15px;'}),
dict(name='div', attrs={'id':'comment-form'})
]

12
recipes/geopolityka.recipe Normal file
View File

@ -0,0 +1,12 @@
from calibre.web.feeds.news import BasicNewsRecipe
class BasicUserRecipe1361379046(BasicNewsRecipe):
title = u'Geopolityka.org'
language = 'pl'
__author__ = 'chemik111'
oldest_article = 15
max_articles_per_feed = 100
auto_cleanup = True
feeds = [(u'Rss', u'http://geopolityka.org/index.php?format=feed&type=rss')]

16
recipes/globe_and_mail.recipe
View File

@ -21,6 +21,10 @@ class AdvancedUserRecipe1287083651(BasicNewsRecipe):
encoding = 'utf8'
publisher = 'Globe & Mail'
language = 'en_CA'
use_embedded_content = False
no_stylesheets = True
auto_cleanup = True
extra_css = 'p.meta {font-size:75%}\n .redtext {color: red;}\n .byline {font-size: 70%}'
feeds = [
@ -44,12 +48,12 @@ class AdvancedUserRecipe1287083651(BasicNewsRecipe):
(re.compile(r'<script.*?</script>', re.DOTALL), lambda m: ''),
]
remove_tags_before = dict(name='h1')
remove_tags = [
dict(name='div', attrs={'id':['ShareArticles', 'topStories']}),
dict(href=lambda x: x and 'tracking=' in x),
{'class':['articleTools', 'pagination', 'Ads', 'topad',
'breadcrumbs', 'footerNav', 'footerUtil', 'downloadlinks']}]
#remove_tags_before = dict(name='h1')
#remove_tags = [
#dict(name='div', attrs={'id':['ShareArticles', 'topStories']}),
#dict(href=lambda x: x and 'tracking=' in x),
#{'class':['articleTools', 'pagination', 'Ads', 'topad',
#'breadcrumbs', 'footerNav', 'footerUtil', 'downloadlinks']}]
def populate_article_metadata(self, article, soup, first):
if first and hasattr(self, 'add_toc_thumbnail'):

66
recipes/hbr.recipe
View File

@ -11,11 +11,11 @@ class HBR(BasicNewsRecipe):
timefmt = ' [%B %Y]'
language = 'en'
no_stylesheets = True
recipe_disabled = ('hbr.org has started requiring the use of javascript'
' to log into their website. This is unsupported in calibre, so'
' this recipe has been disabled. If you would like to see '
' HBR supported in calibre, contact hbr.org and ask them'
' to provide a javascript free login method.')
# recipe_disabled = ('hbr.org has started requiring the use of javascript'
# ' to log into their website. This is unsupported in calibre, so'
# ' this recipe has been disabled. If you would like to see '
# ' HBR supported in calibre, contact hbr.org and ask them'
# ' to provide a javascript free login method.')
LOGIN_URL = 'https://hbr.org/login?request_url=/'
LOGOUT_URL = 'https://hbr.org/logout?request_url=/'
@ -38,46 +38,38 @@ class HBR(BasicNewsRecipe):
#articleAuthors{font-family:Georgia,"Times New Roman",Times,serif; font-style:italic; color:#000000;font-size:x-small;}
#summaryText{font-family:Georgia,"Times New Roman",Times,serif; font-weight:bold; font-size:x-small;}
'''
use_javascript_to_login = True
def get_browser(self):
br = BasicNewsRecipe.get_browser(self)
self.logout_url = None
#'''
br.open(self.LOGIN_URL)
br.select_form(name='signin-form')
br['signin-form:username'] = self.username
br['signin-form:password'] = self.password
raw = br.submit().read()
if '>Sign out<' not in raw:
raise Exception('Failed to login, are you sure your username and password are correct?')
def javascript_login(self, br, username, password):
from calibre.web.jsbrowser.browser import Timeout
try:
link = br.find_link(text='Sign out')
if link:
self.logout_url = link.absolute_url
except:
self.logout_url = self.LOGOUT_URL
#'''
return br
def cleanup(self):
if self.logout_url is not None:
self.browser.open(self.logout_url)
br.visit('https://hbr.org/login?request_url=/', timeout=20)
except Timeout:
pass
br.click('#accordion div[tabindex="0"]', wait_for_load=False)
f = br.select_form('#signin-form')
f['signin-form:username'] = username
f['signin-form:password'] = password
br.submit(wait_for_load=False)
br.run_for_a_time(30)
def map_url(self, url):
if url.endswith('/ar/1'):
return url[:-1]+'pr'
def hbr_get_toc(self):
#return self.index_to_soup(open('/t/hbr.html').read())
# return self.index_to_soup(open('/t/toc.html').read())
today = date.today()
future = today + timedelta(days=30)
for x in [x.strftime('%y%m') for x in (future, today)]:
past = today - timedelta(days=30)
for x in [x.strftime('%y%m') for x in (future, today, past)]:
url = self.INDEX + x
soup = self.index_to_soup(url)
if not soup.find(text='Issue Not Found'):
if (not soup.find(text='Issue Not Found') and not soup.find(
text="We're Sorry. There was an error processing your request")
and 'Exception: java.io.FileNotFoundException' not in
unicode(soup)):
return soup
raise Exception('Could not find current issue')
@ -85,8 +77,9 @@ class HBR(BasicNewsRecipe):
feeds = []
current_section = None
articles = []
for x in soup.find(id='archiveToc').findAll(['h3', 'h4']):
if x.name == 'h3':
for x in soup.find(id='issueFeaturesContent').findAll(['li', 'h4']):
if x.name == 'h4':
if x.get('class', None) == 'basic':continue
if current_section is not None and articles:
feeds.append((current_section, articles))
current_section = self.tag_to_string(x).capitalize()
@ -102,7 +95,7 @@ class HBR(BasicNewsRecipe):
if url.startswith('/'):
url = 'http://hbr.org' + url
url = self.map_url(url)
p = x.parent.find('p')
p = x.find('p', attrs={'class':'author'})
desc = ''
if p is not None:
desc = self.tag_to_string(p)
@ -114,10 +107,9 @@ class HBR(BasicNewsRecipe):
'date':''})
return feeds
def parse_index(self):
soup = self.hbr_get_toc()
#open('/t/hbr.html', 'wb').write(unicode(soup).encode('utf-8'))
# open('/t/hbr.html', 'wb').write(unicode(soup).encode('utf-8'))
feeds = self.hbr_parse_toc(soup)
return feeds

68
recipes/hnonline.recipe Normal file
View File

@ -0,0 +1,68 @@
from calibre.web.feeds.news import BasicNewsRecipe
import re
class HNonlineRecipe(BasicNewsRecipe):
__license__ = 'GPL v3'
__author__ = 'lacike'
language = 'sk'
version = 1
title = u'HNonline'
publisher = u'HNonline'
category = u'News, Newspaper'
description = u'News from Slovakia'
cover_url = u'http://hnonline.sk/img/sk/_relaunch/logo2.png'
oldest_article = 1
max_articles_per_feed = 100
use_embedded_content = False
remove_empty_feeds = True
no_stylesheets = True
remove_javascript = True
# Feeds from: http://rss.hnonline.sk, for listing see http://rss.hnonline.sk/prehlad
feeds = []
feeds.append((u'HNonline|Ekonomika a firmy', u'http://rss.hnonline.sk/?p=kC1000'))
feeds.append((u'HNonline|Slovensko', u'http://rss.hnonline.sk/?p=kC2000'))
feeds.append((u'HNonline|Svet', u'http://rss.hnonline.sk/?p=kC3000'))
feeds.append((u'HNonline|\u0160port', u'http://rss.hnonline.sk/?p=kC4000'))
feeds.append((u'HNonline|Online rozhovor', u'http://rss.hnonline.sk/?p=kCR000'))
feeds.append((u'FinWeb|Spr\u00E1vy zo sveta financi\u00ED', u'http://rss.finweb.hnonline.sk/spravodajstvo'))
feeds.append((u'FinWeb|Koment\u00E1re a anal\u00FDzy', u'http://rss.finweb.hnonline.sk/?p=kPC200'))
feeds.append((u'FinWeb|Invest\u00EDcie', u'http://rss.finweb.hnonline.sk/?p=kPC300'))
feeds.append((u'FinWeb|Svet akci\u00ED', u'http://rss.finweb.hnonline.sk/?p=kPC400'))
feeds.append((u'FinWeb|Rozhovory', u'http://rss.finweb.hnonline.sk/?p=kPC500'))
feeds.append((u'FinWeb|T\u00E9ma t\u00FD\u017Ed\u0148a', u'http://rss.finweb.hnonline.sk/?p=kPC600'))
feeds.append((u'FinWeb|Rebr\u00ED\u010Dky', u'http://rss.finweb.hnonline.sk/?p=kPC700'))
feeds.append((u'HNstyle|Kult\u00FAra', u'http://style.hnonline.sk/?p=kTC100'))
feeds.append((u'HNstyle|Auto-moto', u'http://style.hnonline.sk/?p=kTC200'))
feeds.append((u'HNstyle|Digit\u00E1l', u'http://style.hnonline.sk/?p=kTC300'))
feeds.append((u'HNstyle|Veda', u'http://style.hnonline.sk/?p=kTCV00'))
feeds.append((u'HNstyle|Dizajn', u'http://style.hnonline.sk/?p=kTC400'))
feeds.append((u'HNstyle|Cestovanie', u'http://style.hnonline.sk/?p=kTCc00'))
feeds.append((u'HNstyle|V\u00EDkend', u'http://style.hnonline.sk/?p=kTC800'))
feeds.append((u'HNstyle|Gastro', u'http://style.hnonline.sk/?p=kTC600'))
feeds.append((u'HNstyle|M\u00F3da', u'http://style.hnonline.sk/?p=kTC700'))
feeds.append((u'HNstyle|Modern\u00E1 \u017Eena', u'http://style.hnonline.sk/?p=kTCA00'))
feeds.append((u'HNstyle|Pre\u010Do nie?!', u'http://style.hnonline.sk/?p=k7C000'))
keep_only_tags = []
keep_only_tags.append(dict(name = 'h1', attrs = {'class': 'detail-titulek'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class': 'detail-podtitulek'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class': 'detail-perex'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class': 'detail-text'}))
remove_tags = []
#remove_tags.append(dict(name = 'div', attrs = {'id': re.compile('smeplayer.*')}))
remove_tags_after = []
#remove_tags_after = [dict(name = 'p', attrs = {'class': 'autor_line'})]
extra_css = '''
@font-face {font-family: "serif1";src:url(res:///opt/sony/ebook/FONT/tt0011m_.ttf)}
@font-face {font-family: "sans1";src:url(res:///opt/sony/ebook/FONT/LiberationSans.ttf)}
body {font-family: sans1, serif1;}
'''

BIN
recipes/icons/badania_net.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 968 B

BIN
recipes/icons/elguardian.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 305 B

BIN
recipes/icons/eso_pl.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.6 KiB

BIN
recipes/icons/financial_times_us.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 KiB

BIN
recipes/icons/hnonline.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.5 KiB

BIN
recipes/icons/kurier_galicyjski.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 726 B

BIN
recipes/icons/libertad_digital.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.1 KiB

BIN
recipes/icons/more_intelligent_life.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.6 KiB

BIN
recipes/icons/nauka_w_polsce.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 744 B

BIN
recipes/icons/nezavisne_novine.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 454 B

BIN
recipes/icons/osworld_pl.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.1 KiB

BIN
recipes/icons/pravda_rs.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 606 B

BIN
recipes/icons/ubuntu_pomoc_org.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 757 B

BIN
recipes/icons/wprost_rss.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.7 KiB

12
recipes/informacje_usa.recipe
View File

@ -1,5 +1,4 @@
from calibre.web.feeds.news import BasicNewsRecipe
import re
class Informacje_USA(BasicNewsRecipe):
title = u'Informacje USA'
oldest_article = 7
@ -8,11 +7,10 @@ class Informacje_USA(BasicNewsRecipe):
description = u'portal wiadomości amerykańskich'
category = 'news'
language = 'pl'
masthead_url= 'http://www.informacjeusa.com/wp-content/add_images/top_logo_5_2010.jpg'
cover_url='http://www.informacjeusa.com/wp-content/add_images/top_logo_5_2010.jpg'
cover_url='http://www.informacjeusa.com/wp-content/uploads/2013/01/V3BANNER420-90new.jpg'
no_stylesheets = True
preprocess_regexps = [(re.compile(ur'<p>Zobacz:.*?</p>', re.DOTALL), lambda match: ''), (re.compile(ur'<p><a href=".*?Zobacz także:.*?</a></p>', re.DOTALL), lambda match: ''), (re.compile(ur'<p><p>Zobacz też:.*?</a></p>', re.DOTALL), lambda match: '')]
keep_only_tags=[dict(name='div', attrs={'class':'box box-single'})]
remove_tags_after= dict(attrs={'class':'tags'})
remove_tags= [dict(attrs={'class':['postmetadata', 'tags', 'banner']}), dict(name='a', attrs={'title':['Drukuj', u'Wyślij']})]
use_embedded_content = False
keep_only_tags=[dict(id='post-area')]
remove_tags_after= dict(id='content-area')
remove_tags= [dict(attrs={'class':['breadcrumb']}), dict(id=['social-box', 'social-box-vert'])]
feeds = [(u'Informacje', u'http://www.informacjeusa.com/feed/')]

68
recipes/japan_times.recipe
View File

@ -1,5 +1,5 @@
__license__ = 'GPL v3'
__copyright__ = '2008-2011, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2008-2013, Darko Miletic <darko.miletic at gmail.com>'
'''
japantimes.co.jp
'''
@ -13,59 +13,41 @@ class JapanTimes(BasicNewsRecipe):
language = 'en_JP'
category = 'news, politics, japan'
publisher = 'The Japan Times'
oldest_article = 5
oldest_article = 2
max_articles_per_feed = 150
no_stylesheets = True
use_embedded_content = False
encoding = 'utf8'
publication_type = 'newspaper'
masthead_url = 'http://search.japantimes.co.jp/images/header_title.gif'
masthead_url = 'http://www.japantimes.co.jp/wp-content/themes/jt_theme/library/img/logo-japan-times.png'
extra_css = 'body{font-family: Geneva,Arial,Helvetica,sans-serif}'
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
, 'linearize_tables' : True
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
keep_only_tags = [dict(name='div', attrs={'id':'printresult'})]
remove_tags = [
dict(name=['iframe','meta','link','embed','object','base'])
,dict(attrs={'id':'searchfooter'})
]
feeds = [(u'The Japan Times', u'http://feeds.feedburner.com/japantimes')]
remove_attributes = ['border']
remove_tags_after = dict(name='div', attrs={'class':'entry'})
keep_only_tags = [dict(name='div', attrs={'class':'padding_block'})]
remove_tags = [
dict(name=['iframe','embed','object','base'])
,dict(attrs={'class':['meta_extras','related_articles']})
,dict(attrs={'id':'content_footer_menu'})
]
feeds = [
(u'News' , u'http://www.japantimes.co.jp/news/feed/' )
,(u'Opinion' , u'http://www.japantimes.co.jp/opinion/feed/' )
,(u'Life' , u'http://www.japantimes.co.jp/opinion/feed/' )
,(u'Community', u'http://www.japantimes.co.jp/community/feed/')
,(u'Culture' , u'http://www.japantimes.co.jp/culture/feed/' )
,(u'Sports' , u'http://www.japantimes.co.jp/sports/feed/' )
]
def get_article_url(self, article):
rurl = BasicNewsRecipe.get_article_url(self, article)
return rurl.partition('?')[0]
def print_version(self, url):
if '/rss/' in url:
return url.replace('.jp/rss/','.jp/print/')
if '/text/' in url:
return url.replace('.jp/text/','.jp/print/')
return url
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll('img'):
if not item.has_key('alt'):
item['alt'] = 'image'
for item in soup.findAll('photo'):
item.name = 'div'
for item in soup.head.findAll('paragraph'):
item.extract()
for item in soup.findAll('wwfilename'):
item.extract()
for item in soup.findAll('jtcategory'):
item.extract()
for item in soup.findAll('nomooter'):
item.extract()
for item in soup.body.findAll('paragraph'):
item.name = 'p'
return soup
def preprocess_raw_html(self, raw, url):
return '<html><head>'+raw[raw.find('</head>'):]

14
recipes/kdefamily_pl.recipe Normal file
View File

@ -0,0 +1,14 @@
from calibre.web.feeds.news import BasicNewsRecipe
class KDEFamilyPl(BasicNewsRecipe):
title = u'KDEFamily.pl'
__author__ = 'fenuks'
description = u'KDE w Polsce'
category = 'open source, KDE'
language = 'pl'
cover_url = 'http://www.mykde.home.pl/kdefamily/wp-content/uploads/2012/07/logotype-e1341585198616.jpg'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
use_embedded_content = True
feeds = [(u'Wszystko', u'http://kdefamily.pl/feed/')]

17
recipes/kommersant.recipe
View File

@ -1,5 +1,5 @@
__license__ = 'GPL v3'
__copyright__ = '2010-2011, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2010-2013, Darko Miletic <darko.miletic at gmail.com>'
'''
www.kommersant.ru
'''
@ -29,17 +29,20 @@ class Kommersant_ru(BasicNewsRecipe):
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
keep_only_tags = [dict(attrs={'class':['document','document_vvodka','document_text','document_authors vblock']})]
remove_tags = [dict(name=['iframe','object','link','img','base','meta'])]
feeds = [(u'Articles', u'http://feeds.kommersant.ru/RSS_Export/RU/daily.xml')]
feeds = [(u'Articles', u'http://dynamic.feedsportal.com/pf/438800/http://feeds.kommersant.ru/RSS_Export/RU/daily.xml')]
def get_article_url(self, article):
return article.get('guid', None)
def print_version(self, url):
return url.replace('/doc-rss/','/Doc/') + '/Print'

56
recipes/kurier_galicyjski.recipe Normal file
View File

@ -0,0 +1,56 @@
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import BeautifulSoup as bs
class KurierGalicyjski(BasicNewsRecipe):
title = u'Kurier Galicyjski'
__author__ = 'fenuks'
#description = u''
category = 'news'
language = 'pl'
cover_url = 'http://www.duszki.pl/Kurier_galicyjski_bis2_small.gif'
oldest_article = 7
max_articles_per_feed = 100
remove_empty_feeds = True
no_stylesheets = True
keep_only_tags = [dict(attrs={'class':'item-page'})]
remove_tags = [dict(attrs={'class':'pagenav'}), dict(attrs={'style':'border-top-width: thin; border-top-style: dashed; border-top-color: #CCC; border-bottom-width: thin; border-bottom-style: dashed; border-bottom-color: #CCC; padding-top:5px; padding-bottom:5px; text-align:right; margin-top:10px; height:20px;'})]
feeds = [(u'Wydarzenia', u'http://kuriergalicyjski.com/index.php/wydarzenia?format=feed&type=atom'), (u'Publicystyka', u'http://kuriergalicyjski.com/index.php/niezwykle-historie?format=feed&type=atom'), (u'Reporta\u017ce', u'http://kuriergalicyjski.com/index.php/report?format=feed&type=atom'), (u'Rozmowy Kuriera', u'http://kuriergalicyjski.com/index.php/kuriera?format=feed&type=atom'), (u'Przegl\u0105d prasy', u'http://kuriergalicyjski.com/index.php/2012-01-05-14-08-55?format=feed&type=atom'), (u'Kultura', u'http://kuriergalicyjski.com/index.php/2011-12-02-14-26-39?format=feed&type=atom'), (u'Zabytki', u'http://kuriergalicyjski.com/index.php/2011-12-02-14-27-32?format=feed&type=atom'), (u'Polska-Ukraina', u'http://kuriergalicyjski.com/index.php/pol-ua?format=feed&type=atom'), (u'Polacy i Ukrai\u0144cy', u'http://kuriergalicyjski.com/index.php/polacy-i-ukr?format=feed&type=atom'), (u'Niezwyk\u0142e historie', u'http://kuriergalicyjski.com/index.php/niezwykle-historie?format=feed&type=atom'), (u'Polemiki', u'http://kuriergalicyjski.com/index.php/polemiki?format=feed&type=atom')]
def append_page(self, soup, appendtag):
pager = soup.find(id='article-index')
if pager:
pager = pager.findAll('a')[1:]
if pager:
for a in pager:
nexturl = 'http://www.kuriergalicyjski.com' + a['href']
soup2 = self.index_to_soup(nexturl)
pagetext = soup2.find(attrs={'class':'item-page'})
if pagetext.h2:
pagetext.h2.extract()
r = pagetext.find(attrs={'class':'article-info'})
if r:
r.extract()
pos = len(appendtag.contents)
appendtag.insert(pos, pagetext)
pos = len(appendtag.contents)
for r in appendtag.findAll(id='article-index'):
r.extract()
for r in appendtag.findAll(attrs={'class':'pagenavcounter'}):
r.extract()
for r in appendtag.findAll(attrs={'class':'pagination'}):
r.extract()
for r in appendtag.findAll(attrs={'class':'pagenav'}):
r.extract()
for r in appendtag.findAll(attrs={'style':'border-top-width: thin; border-top-style: dashed; border-top-color: #CCC; border-bottom-width: thin; border-bottom-style: dashed; border-bottom-color: #CCC; padding-top:5px; padding-bottom:5px; text-align:right; margin-top:10px; height:20px;'}):
r.extract()
def preprocess_html(self, soup):
self.append_page(soup, soup.body)
for r in soup.findAll(style=True):
del r['style']
for img in soup.findAll(attrs={'class':'easy_img_caption smartresize'}):
img.insert(len(img.contents)-1, bs('<br />'))
img.insert(len(img.contents), bs('<br /><br />'))
for a in soup.findAll('a', href=True):
if a['href'].startswith('/'):
a['href'] = 'http://kuriergalicyjski.com' + a['href']
return soup

40
recipes/la_nacion_cr.recipe Normal file
View File

@ -0,0 +1,40 @@
from calibre.web.feeds.news import BasicNewsRecipe
class crnews(BasicNewsRecipe):
__author__ = 'Douglas Delgado'
title = u'La Nacion'
publisher = 'GRUPO NACION GN, S. A.'
description = 'Diario de circulacion nacional de Costa Rica. Recipe creado por Douglas Delgado (doudelgado@gmail.com) para su uso con Calibre por Kovid Goyal'
category = 'Spanish, Entertainment'
masthead_url = 'http://www.nacion.com/App_Themes/nacioncom/Images/logo_nacioncom.png'
oldest_article = 7
delay = 1
max_articles_per_feed = 100
auto_cleanup = True
encoding = 'utf-8'
language = 'es_CR'
use_embedded_content = False
remove_empty_feeds = True
remove_javascript = True
no_stylesheets = True
feeds = [(u'Portada', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=portada'), (u'Ultima Hora', u'http://www.nacion.com/Generales/RSS/UltimaHoraRss.aspx'), (u'Nacionales', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=elpais'), (u'Entretenimiento', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=entretenimiento'), (u'Sucesos', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=sucesos'), (u'Deportes', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=deportes'), (u'Internacionales', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=mundo'), (u'Economia', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=economia'), (u'Aldea Global', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=aldeaglobal'), (u'Tecnologia', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=tecnologia'), (u'Opinion', u'http://www.nacion.com/Generales/RSS/EdicionRss.aspx?section=opinion')]
def get_cover_url(self):
index = 'http://kiosko.net/cr/np/cr_nacion.html'
soup = self.index_to_soup(index)
for image in soup.findAll('img',src=True):
if image['src'].endswith('cr_nacion.750.jpg'):
return image['src']
return None
def get_article_url(self, article):
url = article.get('guid', None)
return url
extra_css = '''
h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:30px;}
h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal; font-style:italic; font-size:18px;}
'''

178
recipes/le_monde_sub.recipe
View File

@ -1,132 +1,94 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
__author__ = 'Sylvain Durand <sylvain.durand@ponts.org>'
__license__ = 'GPL v3'
__copyright__ = '2012, Rémi Vanicat <vanicat at debian.org>'
'''
Lemonde.fr: Version abonnée
'''
import os, zipfile, re, time
import time
from calibre import strftime
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import BeautifulSoup
from calibre.ptempfile import PersistentTemporaryFile
from urllib2 import HTTPError
class LeMondeAbonne(BasicNewsRecipe):
class LeMonde(BasicNewsRecipe):
title = u'Le Monde: Édition abonnés'
__author__ = u'Rémi Vanicat'
description = u'Actualités'
category = u'Actualités, France, Monde'
language = 'fr'
needs_subscription = True
title = u'Le Monde: Édition abonnés'
__author__ = 'Sylvain Durand'
description = u'Disponible du lundi au samedi à partir de 14 heures environ, avec tous ses cahiers.'
language = 'fr'
encoding = 'utf8'
no_stylesheets = True
needs_subscription = True
extra_css = u'''
h1{font-size:130%;}
.ariane{font-size:xx-small;}
.source{font-size:xx-small;}
.href{font-size:xx-small;}
.LM_caption{color:#666666; font-size:x-small;}
.main-article-info{font-family:Arial,Helvetica,sans-serif;}
#full-contents{font-size:small; font-family:Arial,Helvetica,sans-serif;font-weight:normal;}
#match-stats-summary{font-size:small; font-family:Arial,Helvetica,sans-serif;font-weight:normal;}
'''
date_url = 'http://www.lemonde.fr/journalelectronique/donnees/libre/%Y%m%d/index.html'
login_url = 'http://www.lemonde.fr/web/journal_electronique/identification/1,56-0,45-0,0.html'
journal_url = 'http://www.lemonde.fr/journalelectronique/donnees/protege/%Y%m%d/%Y%m%d_ipad.xml'
masthead_url = 'http://upload.wikimedia.org/wikipedia/fr/thumb/c/c5/Le_Monde_logo.svg/300px-Le_Monde_logo.svg.png'
couverture_url = 'http://medias.lemonde.fr/abonnes/editionelectronique/%Y%m%d/html/data/img/%y%m%d01.jpg'
zipurl_format = 'http://medias.lemonde.fr/abonnes/editionelectronique/%Y%m%d/html/%y%m%d.zip'
coverurl_format = '/img/%y%m%d01.jpg'
path_format = "%y%m%d"
login_url = 'http://www.lemonde.fr/web/journal_electronique/identification/1,56-0,45-0,0.html'
extra_css = '''
img{max-width:100%}
h1{font-size:1.2em !important; line-height:1.2em !important; }
h2{font-size:1em !important; line-height:1em !important; }
h3{font-size:1em !important; text-transform:uppercase !important; color:#666;}
#photo{text-align:center !important; margin:10px 0 -8px;}
#lgd{font-size:1em !important; line-height:1em !important; font-style:italic; color:#333;} '''
keep_only_tags = [ dict(name="div", attrs={ 'class': 'po-prti' }), dict(name=['h1']), dict(name='div', attrs={ 'class': 'photo' }), dict(name='div', attrs={ 'class': 'po-ti2' }), dict(name='div', attrs={ 'class': 'ar-txt' }), dict(name='div', attrs={ 'class': 'po_rtcol' }) ]
keep_only_tags = [dict(name=['h1','h2','h3','div','txt'])]
article_id_pattern = re.compile("[0-9]+\\.html")
article_url_format = 'http://www.lemonde.fr/journalelectronique/donnees/protege/%Y%m%d/html/'
def __init__(self, options, log, progress_reporter):
BasicNewsRecipe.__init__(self, options, log, progress_reporter)
br = BasicNewsRecipe.get_browser(self)
second = time.time() + 24*60*60
for i in range(7):
self.date = time.gmtime(second)
try:
br.open(time.strftime(self.date_url,self.date))
break
except HTTPError:
second -= 24*60*60
self.timefmt = strftime(u" %A %d %B %Y", self.date).replace(u' 0', u' ')
def get_browser(self):
br = BasicNewsRecipe.get_browser(self)
if self.username is not None and self.password is not None:
br.open(self.login_url)
br.select_form(nr=0)
br['login'] = self.username
br['password'] = self.password
br.submit()
br.open(self.login_url)
br.select_form(nr=0)
br['login'] = self.username
br['password'] = self.password
br.submit()
return br
decalage = 24 * 60 * 60 # today Monde has tomorow date
def get_cover_url(self):
url = time.strftime(self.coverurl_format, self.ltime)
return self.articles_path + url
url = time.strftime(self.couverture_url,self.date)
return url
def parse_index(self):
browser = self.get_browser()
second = time.time()
second += self.decalage
ltime = self.ltime = time.gmtime(second)
url = time.strftime(self.zipurl_format, ltime)
self.timefmt=strftime(" %A %d %B %Y", ltime)
response = browser.open(url)
tmp = PersistentTemporaryFile(suffix='.zip')
self.report_progress(0.1,_('downloading zip file'))
tmp.write(response.read())
tmp.close()
zfile = zipfile.ZipFile(tmp.name, 'r')
self.report_progress(0.1,_('extracting zip file'))
zfile.extractall(self.output_dir)
zfile.close()
path = os.path.join(self.output_dir, time.strftime(self.path_format, ltime), "data")
self.articles_path = path
files = os.listdir(path)
nb_index_files = len([ name for name in files if re.match("frame_gauche_[0-9]+.html", name) ])
flux = []
article_url = time.strftime(self.article_url_format, ltime)
for i in range(nb_index_files):
filename = os.path.join(path, "selection_%d.html" % (i + 1))
tmp = open(filename,'r')
soup=BeautifulSoup(tmp)
title=soup.find('span').contents[0]
tmp.close()
filename = os.path.join(path, "frame_gauche_%d.html" % (i + 1))
tmp = open(filename,'r')
soup = BeautifulSoup(tmp)
url = time.strftime(self.journal_url,self.date)
soup = self.index_to_soup(url).sommaire
sections = []
for sec in soup.findAll("section"):
articles = []
for link in soup.findAll("a"):
article_file = link['href']
article_id=self.article_id_pattern.search(article_file).group()
article = {
'title': link.contents[0],
'url': article_url + article_id,
'descripion': '',
'content': ''
}
articles.append(article)
tmp.close()
if sec['cahier'] != "Le Monde":
for col in sec.findAll("fnts"):
col.extract()
if sec['cahier']=="Le Monde Magazine":
continue
for art in sec.findAll("art"):
if art.txt.string and art.ttr.string:
if art.find(['url']):
art.insert(6,'<div id="photo"><img src="'+art.find(['url']).string+'" /></div>')
if art.find(['lgd']) and art.find(['lgd']).string:
art.insert(7,'<div id="lgd">'+art.find(['lgd']).string+'</div>')
article = "<html><head></head><body>"+unicode(art)+"</body></html>"
article = article.replace('<![CDATA[','').replace(']]>','').replace(' oC ','°C ')
article = article.replace('srttr>','h3>').replace('ssttr>','h2>').replace('ttr>','h1>')
f = PersistentTemporaryFile()
f.write(article)
articles.append({'title':art.ttr.string,'url':"file:///"+f.name})
sections.append((sec['nom'], articles))
return sections
flux.append((title, articles))
return flux
# Local Variables:
# mode: python
# End:
def preprocess_html(self, soup):
for lgd in soup.findAll(id="lgd"):
lgd.contents[-1].extract()
return soup

65
recipes/libertad_digital.recipe Normal file
View File

@ -0,0 +1,65 @@
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
www.libertaddigital.com
'''
from calibre.web.feeds.news import BasicNewsRecipe
class LibertadDigital(BasicNewsRecipe):
title = 'Libertad Digital'
__author__ = 'Darko Miletic'
description = 'En Libertad Digital encontraras noticias y opinion sobre: España, el Mundo, Internet, sociedad, economia y deportes'
publisher = 'Libertad Digital S.A.'
category = 'noticias, ultima hora, españa, internet, mundo, economia, sociedad, Libertad Digital'
oldest_article = 2
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'cp1252'
use_embedded_content = False
language = 'es'
remove_empty_feeds = True
publication_type = 'website'
masthead_url = 'http://s.libertaddigital.com/images/logo.gif'
extra_css = """
body{font-family: Verdana,sans-serif }
img{margin-bottom: 0.4em; display:block}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
remove_tags = [
dict(name=['meta','link','iframe','embed','object'])
,dict(name='p', attrs={'class':'copyright'})
]
remove_attributes=['lang']
feeds = [
(u'Portada' , u'http://feeds2.feedburner.com/libertaddigital/deportes' )
,(u'Opinion' , u'http://feeds2.feedburner.com/libertaddigital/opinion' )
,(u'España' , u'http://feeds2.feedburner.com/libertaddigital/nacional' )
,(u'Internacional', u'http://feeds2.feedburner.com/libertaddigital/internacional')
,(u'Libre Mercado', u'http://feeds2.feedburner.com/libertaddigital/economia' )
,(u'Chic' , u'http://feeds2.feedburner.com/libertaddigital/el-candelabro')
,(u'Internet' , u'http://feeds2.feedburner.com/libertaddigital/internet' )
,(u'Deportes' , u'http://feeds2.feedburner.com/libertaddigital/deportes' )
]
def get_article_url(self, article):
return article.get('guid', None)
def print_version(self, url):
art, sep, rest = url.rpartition('/')
aart, asep, artid = art.rpartition('-')
return 'http://www.libertaddigital.com/c.php?op=imprimir&id=' + artid
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
return soup

18
recipes/mlody_technik_pl.recipe
View File

@ -1,5 +1,5 @@
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
import re
from calibre.web.feeds.news import BasicNewsRecipe
class Mlody_technik(BasicNewsRecipe):
title = u'Młody technik'
@ -9,7 +9,19 @@ class Mlody_technik(BasicNewsRecipe):
language = 'pl'
cover_url='http://science-everywhere.pl/wp-content/uploads/2011/10/mt12.jpg'
no_stylesheets = True
preprocess_regexps = [(re.compile(r"<h4>Podobne</h4>", re.IGNORECASE), lambda m: '')]
oldest_article = 7
max_articles_per_feed = 100
#keep_only_tags=[dict(id='container')]
feeds = [(u'Artyku\u0142y', u'http://www.mt.com.pl/feed')]
remove_empty_feeds = True
use_embedded_content = False
keep_only_tags = [dict(id='content')]
remove_tags = [dict(attrs={'class':'st-related-posts'})]
remove_tags_after = dict(attrs={'class':'entry-content clearfix'})
feeds = [(u'Wszystko', u'http://www.mt.com.pl/feed'),
(u'MT NEWS 24/7', u'http://www.mt.com.pl/kategoria/mt-newsy-24-7/feed'),
(u'Info zoom', u'http://www.mt.com.pl/kategoria/info-zoom/feed'),
(u'm.technik', u'http://www.mt.com.pl/kategoria/m-technik/feed'),
(u'Szkoła', u'http://www.mt.com.pl/kategoria/szkola-2/feed'),
(u'Na Warsztacie', u'http://www.mt.com.pl/kategoria/na-warsztacie/feed'),
(u'Z pasji do...', u'http://www.mt.com.pl/kategoria/z-pasji-do/feed'),
(u'MT testuje', u'http://www.mt.com.pl/kategoria/mt-testuje/feed')]

67
recipes/more_intelligent_life.recipe Normal file
View File

@ -0,0 +1,67 @@
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
moreintelligentlife.com
'''
from calibre.web.feeds.news import BasicNewsRecipe
class MoreIntelligentLife(BasicNewsRecipe):
title = 'More Intelligent Life'
__author__ = 'Darko Miletic'
description = "More Intelligent Life (moreintelligentlife.com) is the online version of Intelligent Life, a lifestyle and culture magazine from The Economist. The website offers not only content from the print edition, trickled out over the course of its shelf-life, but also the Editors' Blog, which carries daily posts from the editorial team-quickfire observations and opinions that allow readers to eavesdrop on the conversation in the office."
publisher = 'The Economist Newspaper ltd'
category = 'arts,lifestyle,intelligent life,the economist,ideas,style,culture'
oldest_article = 60
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'utf8'
use_embedded_content = False
language = 'en'
remove_empty_feeds = True
publication_type = 'website'
extra_css = """
body{font-family: Arial,"Helvetica neue","Bitstream Vera Sans",sans-serif}
img{margin-bottom: 0.4em; display:block}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
keep_only_tags = [dict(attrs={'class':'node'})]
remove_tags_after = dict(attrs={'class':'tags'})
remove_tags = [dict(name=['meta','link','iframe','embed','object'])]
remove_attributes = ['lang']
feeds = [(u'Articles', u'http://feeds.feedburner.com/MoreintelligentlifeTotal')]
def get_cover_url(self):
soup = self.index_to_soup('http://moreintelligentlife.com/')
for image in soup.findAll('img', src=True):
if image['src'].startswith('http://moreintelligentlife.com/files/covers/current_issue_'):
return image['src']
return None
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll('a'):
limg = item.find('img')
if item.string is not None:
str = item.string
item.replaceWith(str)
else:
if limg:
item.name = 'div'
item.attrs = []
else:
str = self.tag_to_string(item)
item.replaceWith(str)
for item in soup.findAll('img'):
if not item.has_key('alt'):
item['alt'] = 'image'
return soup

47
recipes/nauka_w_polsce.recipe Normal file
View File

@ -0,0 +1,47 @@
from calibre.web.feeds.news import BasicNewsRecipe
import re
class NaukawPolsce(BasicNewsRecipe):
title = u'Nauka w Polsce'
__author__ = 'fenuks'
description = u'Serwis Nauka w Polsce ma za zadanie popularyzację polskiej nauki. Można na nim znaleźć wiadomości takie jak: osiągnięcia polskich naukowców, wydarzenia na polskich uczelniach, osiągnięcia studentów, konkursy dla badaczy, staże i stypendia naukowe, wydarzenia w polskiej nauce, kalendarium wydarzeń w nauce, materiały wideo o nauce.'
category = 'science'
language = 'pl'
cover_url = 'http://www.naukawpolsce.pap.pl/Themes/Pap/images/logo-pl.gif'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
remove_empty_feeds = True
index = 'http://www.naukawpolsce.pl'
keep_only_tags = [dict(name='div', attrs={'class':'margines wiadomosc'})]
remove_tags = [dict(name='div', attrs={'class':'tagi'})]
def find_articles(self, url):
articles = []
soup=self.index_to_soup(url)
for i in soup.findAll(name='div', attrs={'class':'aktualnosci-margines lista-depesz information-content'}):
title = i.h1.a.string
url = self.index + i.h1.a['href']
date = '' #i.span.string
articles.append({'title' : title,
'url' : url,
'date' : date,
'description' : ''
})
return articles
def parse_index(self):
feeds = []
feeds.append((u"Historia i kultura", self.find_articles('http://www.naukawpolsce.pl/historia-i-kultura/')))
feeds.append((u"Kosmos", self.find_articles('http://www.naukawpolsce.pl/kosmos/')))
feeds.append((u"Przyroda", self.find_articles('http://www.naukawpolsce.pl/przyroda/')))
feeds.append((u"Społeczeństwo", self.find_articles('http://www.naukawpolsce.pl/spoleczenstwo/')))
feeds.append((u"Technologie", self.find_articles('http://www.naukawpolsce.pl/technologie/')))
feeds.append((u"Uczelnie", self.find_articles('http://www.naukawpolsce.pl/uczelnie/')))
feeds.append((u"Nauki medyczne", self.find_articles('http://www.naukawpolsce.pl/zdrowie/')))
return feeds
def preprocess_html(self, soup):
for p in soup.findAll(name='p', text=re.compile('&nbsp;')):
p.extract()
return soup

40
recipes/navegalo.recipe Normal file
View File

@ -0,0 +1,40 @@
from calibre.web.feeds.news import BasicNewsRecipe
class AdvancedUserRecipe1360354988(BasicNewsRecipe):
title = u'Navegalo.com'
oldest_article = 7
max_articles_per_feed = 100
auto_cleanup = True
from calibre.web.feeds.news import BasicNewsRecipe
class navegalonews(BasicNewsRecipe):
__author__ = 'Douglas Delgado'
title = u'Navegalo.com'
publisher = 'Navegalo.com'
description = 'Noticias actualizadas por Navegalo.com. Recipe creado por Douglas Delgado (doudelgado@gmail.com) para su uso con Calibre'
category = 'Spanish, Entertainment'
masthead_url = 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQZhML5lwsdss6FFF7CFR0Sf-Ln052Zmhs1TlIOcAL8JWN8a-dPlA'
oldest_article = 2
delay = 1
max_articles_per_feed = 100
auto_cleanup = True
encoding = 'utf-8'
language = 'es_CR'
use_embedded_content = False
remove_empty_feeds = True
remove_javascript = True
no_stylesheets = True
feeds = [(u'Nacionales', u'http://www.navegalo.com/es/noticias/noticias/noticias-nacionales?format=feed&type=rss'), (u'Internacionales', u'http://direct.navegalo.com/es/noticias/noticias/noticias-internacionales?format=feed&type=rss'), (u'Deportes', u'http://direct.navegalo.com/es/noticias/noticias/deportes-nacionales?format=feed&type=rss'), (u'Solo futbol', u'http://www.navegalo.com/es/noticias/noticias/solo-futbol?format=feed&type=rss'), (u'Entretenimiento', u'http://www.navegalo.com/es/noticias/noticias/entretenimiento?format=feed&type=rss'), (u'Solo para ellas', u'http://www.navegalo.com/es/noticias/noticias/solo-para-ellas?format=feed&type=rss'), (u'Infiltrados', u'http://direct.navegalo.com/es/noticias/noticias/infiltrados?format=feed&type=rss'), (u'Mano a mano', u'http://direct.navegalo.com/es/noticias/noticias/mano-a-mano?format=feed&type=rss')]
extra_css = '''
h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:30px;}
h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal; font-style:italic; font-size:18px;}
'''

12
recipes/new_yorker.recipe
View File

@ -1,5 +1,5 @@
__license__ = 'GPL v3'
__copyright__ = '2008-2011, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2008-2013, Darko Miletic <darko.miletic at gmail.com>'
'''
newyorker.com
'''
@ -44,20 +44,18 @@ class NewYorker(BasicNewsRecipe):
, 'language' : language
}
keep_only_tags = [
dict(name='div', attrs={'class':'headers'})
,dict(name='div', attrs={'id':['articleheads','items-container','articleRail','articletext','photocredits']})
]
keep_only_tags = [dict(name='div', attrs={'id':'pagebody'})]
remove_tags = [
dict(name=['meta','iframe','base','link','embed','object'])
,dict(attrs={'class':['utils','socialUtils','articleRailLinks','icons'] })
,dict(attrs={'class':['utils','socialUtils','articleRailLinks','icons','social-utils-top','entry-keywords','entry-categories','utilsPrintEmail'] })
,dict(attrs={'id':['show-header','show-footer'] })
]
remove_tags_after = dict(attrs={'class':'entry-content'})
remove_attributes = ['lang']
feeds = [(u'The New Yorker', u'http://www.newyorker.com/services/mrss/feeds/everything.xml')]
def print_version(self, url):
return url + '?printable=true'
return url + '?printable=true&currentPage=all'
def image_url_processor(self, baseurl, url):
return url.strip()

59
recipes/nezavisne_novine.recipe Normal file
View File

@ -0,0 +1,59 @@
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
www.nezavisne.com
'''
from calibre import strftime
from calibre.web.feeds.news import BasicNewsRecipe
class NezavisneNovine(BasicNewsRecipe):
title = 'Nezavisne novine'
__author__ = 'Darko Miletic'
description = 'Nezavisne novine - Najnovije vijesti iz BiH, Srbije, Hrvatske, Crne Gore i svijeta'
publisher = 'NIGP "DNN"'
category = 'news, politics, Bosnia, Balcans'
oldest_article = 2
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'utf8'
use_embedded_content = False
language = 'sr'
remove_empty_feeds = True
publication_type = 'newspaper'
cover_url = strftime('http://pdf.nezavisne.com/slika/novina/nezavisne_novine.jpg?v=%Y%m%d')
masthead_url = 'http://www.nezavisne.com/slika/osnova/nezavisne-novine-logo.gif'
extra_css = """
body{font-family: Arial,Helvetica,sans-serif }
img{margin-bottom: 0.4em; display:block}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
keep_only_tags = [dict(name='div', attrs={'class':'vijest'})]
remove_tags_after = dict(name='div', attrs={'id':'wrap'})
remove_tags = [
dict(name=['meta','link','iframe','object'])
,dict(name='div', attrs={'id':'wrap'})
]
remove_attributes=['lang','xmlns:fb','xmlns:og']
feeds = [
(u'Novosti' , u'http://feeds.feedburner.com/Novosti-NezavisneNovine' )
,(u'Posao' , u'http://feeds.feedburner.com/Posao-NezavisneNovine' )
,(u'Sport' , u'http://feeds.feedburner.com/Sport-NezavisneNovine' )
,(u'Komentar' , u'http://feeds.feedburner.com/Komentari-NezavisneNovine' )
,(u'Umjetnost i zabava' , u'http://feeds.feedburner.com/UmjetnostIZabava-NezavisneNovine' )
,(u'Život i stil' , u'http://feeds.feedburner.com/ZivotIStil-NezavisneNovine' )
,(u'Auto' , u'http://feeds.feedburner.com/Auto-NezavisneNovine' )
,(u'Nauka i tehnologija', u'http://feeds.feedburner.com/NaukaITehnologija-NezavisneNovine')
]
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
return soup

2
recipes/nzherald.recipe
View File

@ -25,7 +25,7 @@ class NewZealandHerald(BasicNewsRecipe):
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000466.xml'),
('Life & Style',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000006.xml'),
('Technology'
('Technology',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000005.xml'),
('Sport',
'http://rss.nzherald.co.nz/rss/xml/nzhrsscid_000000004.xml'),

33
recipes/osworld_pl.recipe Normal file
View File

@ -0,0 +1,33 @@
from calibre.web.feeds.news import BasicNewsRecipe
class OSWorld(BasicNewsRecipe):
title = u'OSWorld.pl'
__author__ = 'fenuks'
description = u'OSWorld.pl to serwis internetowy, dzięki któremu poznasz czym naprawdę jest Open Source. Serwis poświęcony jest wolnemu oprogramowaniu jak linux mint, centos czy ubunty. Znajdziecie u nasz artykuły, unity oraz informacje o certyfikatach CACert. OSWorld to mały świat wielkich systemów!'
category = 'OS, IT, open source, Linux'
language = 'pl'
cover_url = 'http://osworld.pl/wp-content/uploads/osworld-kwadrat-128x111.png'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
remove_empty_feeds = True
use_embedded_content = False
keep_only_tags = [dict(id=['dzial', 'posts'])]
remove_tags = [dict(attrs={'class':'post-comments'})]
remove_tags_after = dict(attrs={'class':'entry clr'})
feeds = [(u'Artyku\u0142y', u'http://osworld.pl/category/artykuly/feed/'), (u'Nowe wersje', u'http://osworld.pl/category/nowe-wersje/feed/')]
def append_page(self, soup, appendtag):
tag = appendtag.find(attrs={'id':'paginacja'})
if tag:
for nexturl in tag.findAll('a'):
soup2 = self.index_to_soup(nexturl['href'])
pagetext = soup2.find(attrs={'class':'entry clr'})
pos = len(appendtag.contents)
appendtag.insert(pos, pagetext)
for r in appendtag.findAll(attrs={'id':'paginacja'}):
r.extract()
def preprocess_html(self, soup):
self.append_page(soup, soup.body)
return soup

60
recipes/pc_lab.recipe
View File

@ -1,5 +1,4 @@
#!/usr/bin/env python
from calibre.web.feeds.recipes import BasicNewsRecipe
class PCLab(BasicNewsRecipe):
@ -8,12 +7,13 @@ class PCLab(BasicNewsRecipe):
__author__ = 'ravcio - rlelusz[at]gmail.com'
description = u"Articles from PC Lab website"
language = 'pl'
oldest_article = 30.0
oldest_article = 30
max_articles_per_feed = 100
recursions = 0
encoding = 'iso-8859-2'
no_stylesheets = True
remove_javascript = True
remove_empty_feeds = True
use_embedded_content = False
keep_only_tags = [
@ -21,50 +21,54 @@ class PCLab(BasicNewsRecipe):
]
remove_tags = [
dict(name='div', attrs={'class':['chapters']})
,dict(name='div', attrs={'id':['script_bxad_slot_display_list_bxad_slot']})
dict(name='div', attrs={'class':['toc first', 'toc', 'tags', 'recommendedarticles', 'name', 'zumi', 'chapters']})
]
remove_tags_after = [
dict(name='div', attrs={'class':['navigation']})
]
#links to RSS feeds
feeds = [ ('PCLab', u'http://pclab.pl/xml/artykuly.xml') ]
feeds = [
(u'Aktualności', 'http://pclab.pl/xml/aktualnosci.xml'),
(u'Artykuły', u'http://pclab.pl/xml/artykuly.xml'),
(u'Poradniki', 'http://pclab.pl/xml/poradniki.xml')
]
#load second and subsequent page content
# in: soup - full page with 'next' button
# out: appendtag - tag to which new page is to be added
def append_page(self, soup, appendtag):
# find the 'Next' button
pager = soup.find('div', attrs={'class':'next'})
pager = soup.find('div', attrs={'class':'navigation'})
if pager:
a = pager.find('a')
if 'news' in a['href']:
pager = None
else:
pager = pager.find('div', attrs={'class':'next'})
while pager:
#search for 'a' element with link to next page (exit if not found)
a = pager.find('a')
if a:
nexturl = a['href']
nexturl = a['href']
soup2 = self.index_to_soup('http://pclab.pl' + nexturl)
pager = soup2.find('div', attrs={'class':'next'})
pagetext = soup2.find('div', attrs={'class':'substance'})
pagetext = pagetext.find('div', attrs={'class':'data'})
soup2 = self.index_to_soup('http://pclab.pl/' + nexturl)
pagetext_substance = soup2.find('div', attrs={'class':'substance'})
pagetext = pagetext_substance.find('div', attrs={'class':'data'})
pagetext.extract()
pos = len(appendtag.contents)
appendtag.insert(pos, pagetext)
pos = len(appendtag.contents)
self.append_page(soup2, appendtag)
pos = len(appendtag.contents)
appendtag.insert(pos, pagetext)
pos = len(appendtag.contents)
pager = soup.find('div', attrs={'class':'navigation'})
if pager:
pager.extract()
def preprocess_html(self, soup):
# soup.body contains no title and no navigator, they are in soup
self.append_page(soup, soup.body)
for link in soup.findAll('a'):
href = link.get('href', None)
if href and href.startswith('/'):
link['href'] = 'http://pclab.pl' + href
# finally remove some tags
tags = soup.findAll('div',attrs={'class':['tags', 'index', 'script_bxad_slot_display_list_bxad_slot', 'index first', 'zumi', 'navigation']})
[tag.extract() for tag in tags]
#for r in soup.findAll('div', attrs={'class':['tags', 'index', 'script_bxad_slot_display_list_bxad_slot', 'index first', 'zumi', 'navigation']})
return soup

55
recipes/pnn.recipe Normal file
View File

@ -0,0 +1,55 @@
from calibre.web.feeds.recipes import BasicNewsRecipe
'''Calibre recipe to convert the RSS feeds of the PNN to an ebook.'''
class SportsIllustratedRecipe(BasicNewsRecipe) :
__author__ = 'n.kucklaender'
__copyright__ = 'a.peter'
__license__ = 'GPL v3'
language = 'de'
description = 'PNN RSS'
version = 1
title = u'PNN'
timefmt = ' [%d.%m.%Y]'
oldest_article = 7.0
no_stylesheets = True
remove_javascript = True
use_embedded_content = False
publication_type = 'newspaper'
remove_empty_feeds = True
remove_tags = [dict(attrs={'class':['um-weather um-header-weather','um-has-sub um-mainnav','um-box','ts-products','um-meta-nav','um-box um-last','um-footer','um-footer-links','share hidden','um-buttons']}),dict(id=['dinsContainer'])]
# remove_tags_before = [dict(name='div', attrs={'class':'um-first'})]
# remove_tags_after = [dict(name='div', attrs={'class':'um-metabar'})]
feeds = [(u'Titelseite', u'http://www.pnn.de/rss.xml'),
(u'Dritte Seite', u'http://www.pnn.de/dritte-seite/rss.xml'),
(u'Politik', u'http://www.pnn.de/politik/rss.xml'),
(u'Meinung', u'http://www.pnn.de/meinung/rss.xml'),
(u'Potsdam', u'http://www.pnn.de/potsdam/rss.xml'),
(u'Havel-Spree', u'http://www.pnn.de/havel-spree/rss.xml'),
(u'Potsdam-Mittelmark', u'http://www.pnn.de/pm/rss.xml'),
(u'Berlin-Brandenburg', u'http://www.pnn.de/brandenburg-berlin/rss.xml'),
(u'Wirtschaft', u'http://www.pnn.de/wirtschaft/rss.xml'),
(u'Sport', u'http://www.pnn.de/sport/rss.xml'),
(u'Regionalsport', u'http://www.pnn.de/regionalsport/rss.xml'),
(u'Kultur', u'http://www.pnn.de/kultur/rss.xml'),
(u'Potsdam-Kultur', u'http://www.pnn.de/potsdam-kultur/rss.xml'),
(u'Wissen', u'http://www.pnn.de/wissen/rss.xml'),
(u'Medien', u'http://www.pnn.de/medien/rss.xml'),
(u'Weltspiegel', u'http://www.pnn.de/weltspiegel/rss.xml'),
(u'Wissenschaft', u'http://www.pnn.de/campus/rss.xml'),
(u'Mobil', u'http://www.pnn.de/mobil/rss.xml'),
(u'Reise', u'http://www.pnn.de/reise/rss.xml'),
(u'Ratgeber', u'http://www.pnn.de/ratgeber/rss.xml'),
(u'Fragen des Tages', u'http://www.pnn.de/fragen-des-tages/rss.xml'),
# (u'Potsdam bin ich', u'http://www.pnn.de/potsdam-bin-ich/rss.xml'),
(u'Leserbriefe', u'http://www.pnn.de/leserbriefe/rss.xml')]
def get_masthead_url(self):
return 'http://www.pnn.de/app/base/img/pnn_logo.png'
def print_version(self, url):
return url.replace('.html', ',view,printVersion.html')

85
recipes/pravda_rs.recipe Normal file
View File

@ -0,0 +1,85 @@
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
__license__ = 'GPL v3'
__copyright__ = '2013, Darko Miletic <darko.miletic at gmail.com>'
'''
www.pravda.rs
'''
import re
from calibre.web.feeds.recipes import BasicNewsRecipe
class Pravda_rs(BasicNewsRecipe):
title = 'Dnevne novine Pravda'
__author__ = 'Darko Miletic'
description = '24 sata portal vesti iz Srbije'
publisher = 'Dnevne novine Pravda'
category = 'news, politics, entertainment, Serbia'
oldest_article = 2
max_articles_per_feed = 100
no_stylesheets = True
encoding = 'utf-8'
use_embedded_content = False
language = 'sr'
publication_type = 'newspaper'
remove_empty_feeds = True
PREFIX = 'http://www.pravda.rs'
FEEDPR = PREFIX + '/category/'
LANGLAT = '?lng=lat'
FEEDSU = '/feed/' + LANGLAT
INDEX = PREFIX + LANGLAT
masthead_url = 'http://www.pravda.rs/wp-content/uploads/2012/09/logoof.png'
extra_css = """
@font-face {font-family: "serif1";src:url(res:///opt/sony/ebook/FONT/tt0011m_.ttf)}
body{font-family: Georgia,"Times New Roman",Times,serif1,serif;}
img{display: block}
"""
conversion_options = {
'comment' : description
, 'tags' : category
, 'publisher': publisher
, 'language' : language
}
preprocess_regexps = [(re.compile(u'\u0110'), lambda match: u'\u00D0')]
keep_only_tags = [dict(name='div', attrs={'class':'post'})]
remove_tags = [dict(name='h3')]
remove_tags_after = dict(name='h3')
feeds = [
(u'Politika' , FEEDPR + 'politika/' + FEEDSU),
(u'Tema Dana', FEEDPR + 'tema-dana/' + FEEDSU),
(u'Hronika' , FEEDPR + 'hronika/' + FEEDSU),
(u'Društvo' , FEEDPR + 'drustvo/' + FEEDSU),
(u'Ekonomija', FEEDPR + 'ekonomija/' + FEEDSU),
(u'Srbija' , FEEDPR + 'srbija/' + FEEDSU),
(u'Beograd' , FEEDPR + 'beograd/' + FEEDSU),
(u'Kultura' , FEEDPR + 'kultura/' + FEEDSU),
(u'Zabava' , FEEDPR + 'zabava/' + FEEDSU),
(u'Sport' , FEEDPR + 'sport/' + FEEDSU),
(u'Svet' , FEEDPR + 'svet/' + FEEDSU),
(u'Porodica' , FEEDPR + 'porodica/' + FEEDSU),
(u'Vremeplov', FEEDPR + 'vremeplov/' + FEEDSU),
(u'IT' , FEEDPR + 'it/' + FEEDSU),
(u'Republika Srpska', FEEDPR + 'republika-srpska/' + FEEDSU),
(u'Crna Gora', FEEDPR + 'crna-gora/' + FEEDSU),
(u'EX YU' , FEEDPR + 'eks-ju/' + FEEDSU),
(u'Dijaspora', FEEDPR + 'dijaspora/' + FEEDSU),
(u'Kolumna' , FEEDPR + 'kolumna/' + FEEDSU),
(u'Afere' , FEEDPR + 'afere/' + FEEDSU),
(u'Feljton' , FEEDPR + 'feljton/' + FEEDSU),
(u'Intervju' , FEEDPR + 'intervju/' + FEEDSU),
(u'Reportaža', FEEDPR + 'reportaza/' + FEEDSU),
(u'Zanimljivosti', FEEDPR + 'zanimljivosti/' + FEEDSU),
(u'Sa trga' , FEEDPR + 'sa-trga/' + FEEDSU)
]
def print_version(self, url):
return url + self.LANGLAT
def preprocess_raw_html(self, raw, url):
return '<html><head><title>title</title>'+raw[raw.find('</head>'):]

33
recipes/revista_cromos.recipe Normal file
View File

@ -0,0 +1,33 @@
# coding=utf-8
# https://github.com/iemejia/calibrecolombia
'''
http://www.cromos.com.co/
'''
from calibre.web.feeds.news import BasicNewsRecipe
class ElMalpensante(BasicNewsRecipe):
title = u'Revista Cromos'
language = 'es_CO'
__author__ = 'Ismael Mejia <iemejia@gmail.com>'
cover_url = 'http://www.cromos.com.co/sites/cromos.com.co/themes/cromos_theme/images/logo_morado.gif'
description = 'Revista Cromos'
oldest_article = 7
simultaneous_downloads = 20
#tags = 'news, sport, blog'
use_embedded_content = True
remove_empty_feeds = True
max_articles_per_feed = 100
feeds = [(u'Cromos', u'http://www.cromos.com.co/rss.xml'),
(u'Moda', u'http://www.cromos.com.co/moda/feed'),
(u'Estilo de Vida', u'http://www.cromos.com.co/estilo-de-vida/feed'),
(u'Cuidado Personal', u'http://www.cromos.com.co/estilo-de-vida/cuidado-personal/feed'),
(u'Salud y Alimentación', u'http://www.cromos.com.co/estilo-de-vida/salud-y-alimentacion/feed'),
(u'Personajes', u'http://www.cromos.com.co/personajes/feed'),
(u'Actualidad', u'http://www.cromos.com.co/personajes/actualidad/feed'),
(u'Espectáculo', u'http://www.cromos.com.co/personajes/espectaculo/feed'),
(u'Reportajes', u'http://www.cromos.com.co/reportajes/feed'),
(u'Eventos', u'http://www.cromos.com.co/eventos/feed'),
(u'Modelos', u'http://www.cromos.com.co/modelos/feed'),
]

61
recipes/science_news.recipe
View File

@ -1,24 +1,38 @@
#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2008, Darko Miletic <darko.miletic at gmail.com>'
'''
sciencenews.org
'''
from calibre.web.feeds.news import BasicNewsRecipe
class Sciencenews(BasicNewsRecipe):
title = u'ScienceNews'
__author__ = u'Darko Miletic and Sujata Raman'
description = u"Science News is an award-winning weekly newsmagazine covering the most important research in all fields of science. Its 16 pages each week are packed with short, accurate articles that appeal to both general readers and scientists. Published since 1922, the magazine now reaches about 150,000 subscribers and more than 1 million readers. These are the latest News Items from Science News."
class ScienceNewsIssue(BasicNewsRecipe):
title = u'Science News Recent Issues'
__author__ = u'Darko Miletic, Sujata Raman and Starson17'
description = u'''Science News is an award-winning weekly
newsmagazine covering the most important research in all fields of science.
Its 16 pages each week are packed with short, accurate articles that appeal
to both general readers and scientists. Published since 1922, the magazine
now reaches about 150,000 subscribers and more than 1 million readers.
These are the latest News Items from Science News. This recipe downloads
the last 30 days worth of articles.'''
category = u'Science, Technology, News'
publisher = u'Society for Science & the Public'
oldest_article = 30
language = 'en'
max_articles_per_feed = 100
no_stylesheets = True
use_embedded_content = False
auto_cleanup = True
timefmt = ' [%A, %d %B, %Y]'
recursions = 1
remove_attributes = ['style']
conversion_options = {'linearize_tables' : True
, 'comment' : description
, 'tags' : category
, 'publisher' : publisher
, 'language' : language
}
extra_css = '''
.content_description{font-family:georgia ;font-size:x-large; color:#646464 ; font-weight:bold;}
@ -27,36 +41,33 @@ class Sciencenews(BasicNewsRecipe):
.content_edition{font-family:helvetica,arial ;font-size: xx-small ;}
.exclusive{color:#FF0000 ;}
.anonymous{color:#14487E ;}
.content_content{font-family:helvetica,arial ;font-size: x-small ; color:#000000;}
.description{color:#585858;font-family:helvetica,arial ;font-size: xx-small ;}
.content_content{font-family:helvetica,arial ;font-size: medium ; color:#000000;}
.description{color:#585858;font-family:helvetica,arial ;font-size: large ;}
.credit{color:#A6A6A6;font-family:helvetica,arial ;font-size: xx-small ;}
'''
#keep_only_tags = [ dict(name='div', attrs={'id':'column_action'}) ]
#remove_tags_after = dict(name='ul', attrs={'id':'content_functions_bottom'})
#remove_tags = [
#dict(name='ul', attrs={'id':'content_functions_bottom'})
#,dict(name='div', attrs={'id':['content_functions_top','breadcrumb_content']})
#,dict(name='img', attrs={'class':'icon'})
#,dict(name='div', attrs={'class': 'embiggen'})
#]
keep_only_tags = [ dict(name='div', attrs={'class':'content_content'}),
dict(name='ul', attrs={'id':'toc'})
]
feeds = [(u"Science News / News Items", u'http://sciencenews.org/index.php/feed/type/news/name/news.rss/view/feed/name/all.rss')]
feeds = [(u"Science News Current Issues", u'http://www.sciencenews.org/view/feed/type/edition/name/issues.rss')]
match_regexps = [
r'www.sciencenews.org/view/feature/id/',
r'www.sciencenews.org/view/generic/id'
]
def get_cover_url(self):
cover_url = None
index = 'http://www.sciencenews.org/view/home'
soup = self.index_to_soup(index)
link_item = soup.find(name = 'img',alt = "issue")
print link_item
if link_item:
cover_url = 'http://www.sciencenews.org' + link_item['src'] + '.jpg'
return cover_url
#def preprocess_html(self, soup):
#for tag in soup.findAll(name=['span']):
#tag.name = 'div'
#return soup
def preprocess_html(self, soup):
for tag in soup.findAll(name=['span']):
tag.name = 'div'
return soup

40
recipes/scmp.recipe
View File

@ -4,7 +4,6 @@ __copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
scmp.com
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe
class SCMP(BasicNewsRecipe):
@ -18,10 +17,11 @@ class SCMP(BasicNewsRecipe):
max_articles_per_feed = 200
no_stylesheets = True
encoding = 'utf-8'
auto_cleanup = True
use_embedded_content = False
language = 'en_CN'
remove_empty_feeds = True
needs_subscription = True
needs_subscription = 'optional'
publication_type = 'newspaper'
masthead_url = 'http://www.scmp.com/images/logo_scmp_home.gif'
extra_css = ' body{font-family: Arial,Helvetica,sans-serif } '
@ -46,17 +46,17 @@ class SCMP(BasicNewsRecipe):
br.submit()
return br
remove_attributes=['width','height','border']
#remove_attributes=['width','height','border']
keep_only_tags = [
dict(attrs={'id':['ART','photoBox']})
,dict(attrs={'class':['article_label','article_byline','article_body']})
]
#keep_only_tags = [
#dict(attrs={'id':['ART','photoBox']})
#,dict(attrs={'class':['article_label','article_byline','article_body']})
#]
preprocess_regexps = [
(re.compile(r'<P><table((?!<table).)*class="embscreen"((?!</table>).)*</table>', re.DOTALL|re.IGNORECASE),
lambda match: ''),
]
#preprocess_regexps = [
#(re.compile(r'<P><table((?!<table).)*class="embscreen"((?!</table>).)*</table>', re.DOTALL|re.IGNORECASE),
#lambda match: ''),
#]
feeds = [
(u'Business' , u'http://www.scmp.com/rss/business.xml' )
@ -68,13 +68,13 @@ class SCMP(BasicNewsRecipe):
,(u'Sport' , u'http://www.scmp.com/rss/sport.xml' )
]
def print_version(self, url):
rpart, sep, rest = url.rpartition('&')
return rpart #+ sep + urllib.quote_plus(rest)
#def print_version(self, url):
#rpart, sep, rest = url.rpartition('&')
#return rpart #+ sep + urllib.quote_plus(rest)
def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
items = soup.findAll(src="/images/label_icon.gif")
[item.extract() for item in items]
return self.adeify_images(soup)
#def preprocess_html(self, soup):
#for item in soup.findAll(style=True):
#del item['style']
#items = soup.findAll(src="/images/label_icon.gif")
#[item.extract() for item in items]
#return self.adeify_images(soup)

1
recipes/seattle_times.recipe
View File

@ -23,6 +23,7 @@ class SeattleTimes(BasicNewsRecipe):
language = 'en'
auto_cleanup = True
auto_cleanup_keep = '//div[@id="PhotoContainer"]'
cover_url = 'http://seattletimes.com/PDF/frontpage.pdf'
feeds = [
(u'Top Stories',

9
recipes/spiders_web_pl.recipe
View File

@ -5,11 +5,14 @@ class SpidersWeb(BasicNewsRecipe):
oldest_article = 7
__author__ = 'fenuks'
description = u''
cover_url = 'http://www.spidersweb.pl/wp-content/themes/spiderweb/img/Logo.jpg'
cover_url = 'http://www.spidersweb.pl/wp-content/themes/new_sw/images/spidersweb.png'
category = 'IT, WEB'
language = 'pl'
no_stylesheers=True
remove_javascript = True
use_embedded_content = False
max_articles_per_feed = 100
keep_only_tags=[dict(id='Post')]
remove_tags=[dict(name='div', attrs={'class':['Comments', 'Shows', 'Post-Tags']}), dict(id='Author-Column')]
keep_only_tags=[dict(id='start')]
remove_tags_after = dict(attrs={'class':'padding20'})
remove_tags=[dict(name='div', attrs={'class':['padding border-bottom', 'padding20', 'padding border-top']})]
feeds = [(u'Wpisy', u'http://www.spidersweb.pl/feed')]

34
recipes/thestar.recipe
View File

@ -1,7 +1,5 @@
#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2009, Darko Miletic <darko.miletic at gmail.com>'
__copyright__ = '2009-2013, Darko Miletic <darko.miletic at gmail.com>'
'''
www.thestar.com
'''
@ -11,18 +9,17 @@ from calibre.web.feeds.news import BasicNewsRecipe
class TheTorontoStar(BasicNewsRecipe):
title = 'The Toronto Star'
__author__ = 'Darko Miletic'
description = "Canada's largest daily newspaper"
description = "Thestar.com is Canada's largest online news site. Stay current with our sports, business entertainment news and more from the Toronto Star and thestar.com"
oldest_article = 2
language = 'en_CA'
max_articles_per_feed = 100
no_stylesheets = True
#auto_cleanup = True
#auto_cleanup_keep = '//div[@class="topsContent topsContentActive"]'
use_embedded_content = False
delay = 2
publisher = 'The Toronto Star'
category = "Toronto Star,Canada's largest daily newspaper,breaking news,classifieds,careers,GTA,Toronto Maple Leafs,sports,Toronto,news,editorial,The Star,Ontario,information,columnists,business,entertainment,births,deaths,automotive,rentals,weather,archives,Torstar,technology,Joseph Atkinson"
encoding = 'utf-8'
masthead_url = 'http://www.thestar.com/etc/designs/thestar/images/general/logoLrg.png'
conversion_options = {
'comments' : description
@ -30,23 +27,18 @@ class TheTorontoStar(BasicNewsRecipe):
,'publisher' : publisher
}
#keep_only_tags = [dict(name='div', attrs={'class':'ts-article'})]
#remove_tags_before = dict(name='div',attrs={'id':'ts-article_header'})
remove_tags_before = dict(name='div',attrs={'class':'article-headline'})
feeds = [
(u'News' , u'http://www.thestar.com/rss/?categories=293' )
,(u'Opinion' , u'http://www.thestar.com/rss/?categories=303' )
,(u'Business' , u'http://www.thestar.com/rss/?categories=294' )
,(u'Sports' , u'http://www.thestar.com/rss/?categories=295' )
,(u'Entertainment', u'http://www.toronto.com/rss?categories=6298' )
,(u'Living' , u'http://www.thestar.com/rss/?categories=297' )
,(u'Travel' , u'http://www.thestar.com/rss/list/1042246?' )
,(u'Science' , u'http://www.thestar.com/rss?categories=6481')
(u'News' , u'http://www.thestar.com/feeds.articles.news.rss' )
,(u'Opinion' , u'http://www.thestar.com/feeds.articles.opinion.rss' )
,(u'Business' , u'http://www.thestar.com/feeds.articles.business.rss' )
,(u'Sports' , u'http://www.thestar.com/feeds.articles.sports.rss' )
,(u'Entertainment', u'http://www.thestar.com/feeds.articles.entertainment.rss' )
,(u'Living' , u'http://www.thestar.com/feeds.articles.life.rss' )
,(u'Travel' , u'http://www.thestar.com/feeds.articles.life.travel.rss' )
,(u'Technology' , u'http://www.thestar.com/feeds.articles.life.technology.rss')
]
def print_version(self, url):
artl = url.rpartition('--')[0]
artid = artl.rpartition('/')[2]
return 'http://www.thestar.com/printarticle/' + artid
return url.replace('.html', '.print.html')

22
recipes/ubuntu_pomoc_org.recipe Normal file
View File

@ -0,0 +1,22 @@
import re
from calibre.web.feeds.news import BasicNewsRecipe
class UbuntuPomoc(BasicNewsRecipe):
title = u'Ubuntu-pomoc.org'
__author__ = 'fenuks'
description = u'Strona poświęcona systemowi Ubuntu Linux. Znajdziesz tutaj przydatne i sprawdzone poradniki oraz sposoby rozwiązywania wielu popularnych problemów. Ten blog rozwiąże każdy Twój problem - jeśli nie teraz, to wkrótce! :)'
category = 'Linux, Ubuntu, open source'
language = 'pl'
cover_url = 'http://www.ubuntu-pomoc.org/grafika/ubuntupomoc.png'
preprocess_regexps = [(re.compile(r'<div class="ciekawostka">.+', re.IGNORECASE|re.DOTALL), lambda m: '')]
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
remove_javascript = True
remove_empty_feeds = True
use_embedded_content = False
remove_attrs = ['style']
keep_only_tags = [dict(attrs={'class':'post'})]
remove_tags_after = dict(attrs={'class':'underEntry'})
remove_tags = [dict(attrs={'class':['underPostTitle', 'yarpp-related', 'underEntry', 'social', 'tags', 'commentlist', 'youtube_sc']}), dict(id=['wp_rp_first', 'commentReply'])]
feeds = [(u'Ca\u0142o\u015b\u0107', u'http://feeds.feedburner.com/Ubuntu-Pomoc'),
(u'Gry', u'http://feeds.feedburner.com/GryUbuntu-pomoc')]

22
recipes/unperiodico.recipe Normal file
View File

@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
# https://github.com/iemejia/calibrecolombia
'''
http://www.unperiodico.unal.edu.co/
'''
from calibre import strftime
from calibre.web.feeds.news import BasicNewsRecipe
class UNPeriodico(BasicNewsRecipe):
title = u'UN Periodico'
language = 'es_CO'
__author__ = 'Ismael Mejia <iemejia@gmail.com>'
cover_url = 'http://www.unperiodico.unal.edu.co/fileadmin/templates/periodico/img/logoperiodico.png'
description = 'UN Periodico'
oldest_article = 30
max_articles_per_feed = 100
publication_type = 'newspaper'
feeds = [
(u'UNPeriodico', u'http://www.unperiodico.unal.edu.co/rss/type/rss2/')
]

253
recipes/vic_times.recipe
View File

@ -1,105 +1,46 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
__license__ = 'GPL v3'
'''
www.canada.com
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import BeautifulStoneSoup
from calibre.web.feeds.recipes import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import Tag, BeautifulStoneSoup
class CanWestPaper(BasicNewsRecipe):
class TimesColonist(BasicNewsRecipe):
# un-comment the following four lines for the Victoria Times Colonist
title = u'Victoria Times Colonist'
url_prefix = 'http://www.timescolonist.com'
description = u'News from Victoria, BC'
fp_tag = 'CAN_TC'
# un-comment the following four lines for the Vancouver Province
## title = u'Vancouver Province'
## url_prefix = 'http://www.theprovince.com'
## description = u'News from Vancouver, BC'
## fp_tag = 'CAN_VP'
# un-comment the following four lines for the Vancouver Sun
## title = u'Vancouver Sun'
## url_prefix = 'http://www.vancouversun.com'
## description = u'News from Vancouver, BC'
## fp_tag = 'CAN_VS'
# un-comment the following four lines for the Edmonton Journal
## title = u'Edmonton Journal'
## url_prefix = 'http://www.edmontonjournal.com'
## description = u'News from Edmonton, AB'
## fp_tag = 'CAN_EJ'
# un-comment the following four lines for the Calgary Herald
## title = u'Calgary Herald'
## url_prefix = 'http://www.calgaryherald.com'
## description = u'News from Calgary, AB'
## fp_tag = 'CAN_CH'
# un-comment the following four lines for the Regina Leader-Post
## title = u'Regina Leader-Post'
## url_prefix = 'http://www.leaderpost.com'
## description = u'News from Regina, SK'
## fp_tag = ''
# un-comment the following four lines for the Saskatoon Star-Phoenix
## title = u'Saskatoon Star-Phoenix'
## url_prefix = 'http://www.thestarphoenix.com'
## description = u'News from Saskatoon, SK'
## fp_tag = ''
# un-comment the following four lines for the Windsor Star
## title = u'Windsor Star'
## url_prefix = 'http://www.windsorstar.com'
## description = u'News from Windsor, ON'
## fp_tag = 'CAN_'
# un-comment the following four lines for the Ottawa Citizen
## title = u'Ottawa Citizen'
## url_prefix = 'http://www.ottawacitizen.com'
## description = u'News from Ottawa, ON'
## fp_tag = 'CAN_OC'
# un-comment the following four lines for the Montreal Gazette
## title = u'Montreal Gazette'
## url_prefix = 'http://www.montrealgazette.com'
## description = u'News from Montreal, QC'
## fp_tag = 'CAN_MG'
url_list = []
language = 'en_CA'
__author__ = 'Nick Redding'
no_stylesheets = True
timefmt = ' [%b %d]'
timefmt = ' [%b %d]'
encoding = 'utf-8'
extra_css = '''
.timestamp { font-size:xx-small; display: block; }
#storyheader { font-size: medium; }
#storyheader h1 { font-size: x-large; }
#storyheader h2 { font-size: large; font-style: italic; }
.byline { font-size:xx-small; }
#photocaption { font-size: small; font-style: italic }
#photocredit { font-size: xx-small; }'''
keep_only_tags = [dict(name='div', attrs={'id':'storyheader'}),dict(name='div', attrs={'id':'storycontent'})]
.byline { font-size:xx-small; font-weight: bold;}
h3 { margin-bottom: 6px; }
.caption { font-size: xx-small; font-style: italic; font-weight: normal; }
'''
keep_only_tags = [dict(name='div', attrs={'class':re.compile('main.content')})]
remove_tags = [{'class':'comments'},
dict(name='div', attrs={'class':'navbar'}),dict(name='div', attrs={'class':'morelinks'}),
dict(name='div', attrs={'class':'viewmore'}),dict(name='li', attrs={'class':'email'}),
dict(name='div', attrs={'class':'story_tool_hr'}),dict(name='div', attrs={'class':'clear'}),
dict(name='div', attrs={'class':'story_tool'}),dict(name='div', attrs={'class':'copyright'}),
dict(name='div', attrs={'class':'rule_grey_solid'}),
dict(name='li', attrs={'class':'print'}),dict(name='li', attrs={'class':'share'}),dict(name='ul', attrs={'class':'bullet'})]
{'id':'photocredit'},
dict(name='div', attrs={'class':re.compile('top.controls')}),
dict(name='div', attrs={'class':re.compile('social')}),
dict(name='div', attrs={'class':re.compile('tools')}),
dict(name='div', attrs={'class':re.compile('bottom.tools')}),
dict(name='div', attrs={'class':re.compile('window')}),
dict(name='div', attrs={'class':re.compile('related.news.element')})]
def get_cover_url(self):
from datetime import timedelta, date
if self.fp_tag=='':
return None
cover = 'http://webmedia.newseum.org/newseum-multimedia/dfp/jpg'+str(date.today().day)+'/lg/'+self.fp_tag+'.jpg'
br = BasicNewsRecipe.get_browser(self)
daysback=1
@ -120,6 +61,18 @@ class CanWestPaper(BasicNewsRecipe):
cover = None
return cover
def prepare_masthead_image(self, path_to_image, out_path):
if self.Kindle_Fire:
from calibre.utils.magick import Image, create_canvas
img = Image()
img.open(path_to_image)
width, height = img.size
img2 = create_canvas(width, height)
img2.compose(img)
img2.save(out_path)
else:
BasicNewsRecipe.prepare_masthead_image(path_to_image, out_path)
def fixChars(self,string):
# Replace lsquo (\x91)
fixed = re.sub("\x91","‘",string)
@ -166,55 +119,107 @@ class CanWestPaper(BasicNewsRecipe):
a.replaceWith(a.renderContents().decode('cp1252','replace'))
return soup
def preprocess_html(self, soup):
def preprocess_html(self,soup):
byline = soup.find('p',attrs={'class':re.compile('ancillary')})
if byline is not None:
byline.find('a')
authstr = self.tag_to_string(byline,False)
authstr = re.sub('/ *Times Colonist','/',authstr, flags=re.IGNORECASE)
authstr = re.sub('BY */','',authstr, flags=re.IGNORECASE)
newdiv = Tag(soup,'div')
newdiv.insert(0,authstr)
newdiv['class']='byline'
byline.replaceWith(newdiv)
for caption in soup.findAll('p',attrs={'class':re.compile('caption')}):
capstr = self.tag_to_string(caption,False)
capstr = re.sub('Photograph by.*$','',capstr, flags=re.IGNORECASE)
newdiv = Tag(soup,'div')
newdiv.insert(0,capstr)
newdiv['class']='caption'
caption.replaceWith(newdiv)
for ptag in soup.findAll('p'):
ptext = self.tag_to_string(ptag,use_alt=False, normalize_whitespace=True)
ptext = re.sub(r'\s+','', ptext)
if (ptext=='') or (ptext=='&nbsp;'):
ptag.extract()
return self.strip_anchors(soup)
raeside = False
def handle_articles(self,htag,article_list,sectitle):
atag = htag.a
if atag is not None:
url = atag['href']
#print("Checking "+url)
if atag['href'].startswith('/'):
url = self.url_prefix+atag['href']
if url in self.url_list:
return
self.url_list.append(url)
title = self.tag_to_string(atag,False)
if 'VIDEO' in title.upper():
return
if 'GALLERY' in title.upper():
return
if 'PHOTOS' in title.upper():
return
if 'RAESIDE' in title.upper():
if self.raeside:
return
self.raeside = True
dtag = htag.findNext('p')
description=''
if dtag is not None:
description = self.tag_to_string(dtag,False)
article_list.append(dict(title=title,url=url,date='',description=description,author='',content=''))
#print(sectitle+title+": description = "+description+" URL="+url)
def add_section_index(self,ans,securl,sectitle):
print("Add section url="+self.url_prefix+'/'+securl)
try:
soup = self.index_to_soup(self.url_prefix+'/'+securl)
except:
return ans
mainsoup = soup.find('div',attrs={'class':re.compile('main.content')})
article_list = []
for wdiv in mainsoup.findAll('div',attrs={'id':re.compile('featured.story')}):
for htag in wdiv.findAll('h3'):
self.handle_articles(htag,article_list,sectitle)
for ladiv in mainsoup.findAll(attrs={'class':re.compile('leading.articles')}):
for wdiv in mainsoup.findAll('div',attrs={'class':re.compile('article.row')}):
for htag in wdiv.findAll('h2'):
self.handle_articles(htag,article_list,sectitle)
ans.append((sectitle,article_list))
return ans
def parse_index(self):
soup = self.index_to_soup(self.url_prefix+'/news/todays-paper/index.html')
articles = {}
key = 'News'
ans = ['News']
# Find each instance of class="sectiontitle", class="featurecontent"
for divtag in soup.findAll('div',attrs={'class' : ["section_title02","featurecontent"]}):
#self.log(" div class = %s" % divtag['class'])
if divtag['class'].startswith('section_title'):
# div contains section title
if not divtag.h3:
continue
key = self.tag_to_string(divtag.h3,False)
ans.append(key)
self.log("Section name %s" % key)
continue
# div contains article data
h1tag = divtag.find('h1')
if not h1tag:
continue
atag = h1tag.find('a',href=True)
if not atag:
continue
url = self.url_prefix+'/news/todays-paper/'+atag['href']
#self.log("Section %s" % key)
#self.log("url %s" % url)
title = self.tag_to_string(atag,False)
#self.log("title %s" % title)
pubdate = ''
description = ''
ptag = divtag.find('p');
if ptag:
description = self.tag_to_string(ptag,False)
#self.log("description %s" % description)
author = ''
autag = divtag.find('h4')
if autag:
author = self.tag_to_string(autag,False)
#self.log("author %s" % author)
if not articles.has_key(key):
articles[key] = []
articles[key].append(dict(title=title,url=url,date=pubdate,description=description,author=author,content=''))
ans = [(key, articles[key]) for key in ans if articles.has_key(key)]
ans = []
ans = self.add_section_index(ans,'','Web Front Page')
ans = self.add_section_index(ans,'news/','News Headlines')
ans = self.add_section_index(ans,'news/b-c/','BC News')
ans = self.add_section_index(ans,'news/national/','Natioanl News')
ans = self.add_section_index(ans,'news/world/','World News')
ans = self.add_section_index(ans,'opinion/','Opinion')
ans = self.add_section_index(ans,'opinion/letters/','Letters')
ans = self.add_section_index(ans,'business/','Business')
ans = self.add_section_index(ans,'business/money/','Money')
ans = self.add_section_index(ans,'business/technology/','Technology')
ans = self.add_section_index(ans,'business/working/','Working')
ans = self.add_section_index(ans,'sports/','Sports')
ans = self.add_section_index(ans,'sports/hockey/','Hockey')
ans = self.add_section_index(ans,'sports/football/','Football')
ans = self.add_section_index(ans,'sports/basketball/','Basketball')
ans = self.add_section_index(ans,'sports/golf/','Golf')
ans = self.add_section_index(ans,'entertainment/','entertainment')
ans = self.add_section_index(ans,'entertainment/go/','Go!')
ans = self.add_section_index(ans,'entertainment/music/','Music')
ans = self.add_section_index(ans,'entertainment/books/','Books')
ans = self.add_section_index(ans,'entertainment/Movies/','movies')
ans = self.add_section_index(ans,'entertainment/television/','Television')
ans = self.add_section_index(ans,'life/','Life')
ans = self.add_section_index(ans,'life/health/','Health')
ans = self.add_section_index(ans,'life/travel/','Travel')
ans = self.add_section_index(ans,'life/driving/','Driving')
ans = self.add_section_index(ans,'life/homes/','Homes')
ans = self.add_section_index(ans,'life/food-drink/','Food & Drink')
return ans

40
recipes/vice_magazine_de.recipe Normal file
View File

@ -0,0 +1,40 @@
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
import re
from calibre.web.feeds.news import BasicNewsRecipe
class ViceDERecipe(BasicNewsRecipe):
title = u'Vice Magazin Deutschland'
__author__ = 'atordo;alex'
description = u'Die offizielle Website des Vice Magazins Deutschland'
category = u'Nachrichten, Fotografie, Blogs, Mode, Kunst, Film, Musik, Literatur, Technik'
cover_url = 'http://www.seeklogo.com/images/V/Vice-logo-668578AC94-seeklogo.com.gif'
oldest_article = 14
max_articles_per_feed = 100
auto_cleanup = False
no_stylesheets = True
language = 'de'
use_embedded_content = False
remove_javascript = True
publication_type = 'magazine'
recursions=10
match_regexps = [r'/read/.*\?Contentpage=[2-9]$']
keep_only_tags = [
dict(attrs={'class':['article_title','article_content','next']})
]
remove_tags = [
dict(attrs={'class':['social_buttons','search','tweet','like','inline_socials'
,'stumblebadge','plusone']})
]
extra_css = '''
.author{font-size:small}
img{margin-bottom: 0.4em; display:block; margin-left:auto; margin-right: auto}
'''
preprocess_regexps = [
(re.compile(r'<img src="http://.*\.scorecardresearch\.com/'), lambda m: '')
]
feeds = [('Vice', 'http://www.vice.com/de/rss')]

4
recipes/volksrant.recipe
View File

@ -9,7 +9,6 @@ __docformat__ = 'restructuredtext en'
Modified by Tony Stegall
on 10/10/10 to include function to grab print version of articles
'''
from datetime import date
from calibre.web.feeds.news import BasicNewsRecipe
'''
@ -44,8 +43,7 @@ class AdvancedUserRecipe1249039563(BasicNewsRecipe):
articles_are_obfuscated = True
def get_obfuscated_article(self, url):
br = self.get_browser()
print 'THE CURRENT URL IS: ', url
br = self.browser.clone_browser()
br.open(url)
year = date.today().year

142
recipes/wprost.recipe
View File

@ -10,89 +10,89 @@ from calibre.web.feeds.news import BasicNewsRecipe
import re
class Wprost(BasicNewsRecipe):
EDITION = 0
FIND_LAST_FULL_ISSUE = True
EXCLUDE_LOCKED = True
ICO_BLOCKED = 'http://www.wprost.pl/G/layout2/ico_blocked.png'
EDITION = 0
FIND_LAST_FULL_ISSUE = True
EXCLUDE_LOCKED = True
ICO_BLOCKED = 'http://www.wprost.pl/G/layout2/ico_blocked.png'
title = u'Wprost'
__author__ = 'matek09'
description = 'Weekly magazine'
encoding = 'ISO-8859-2'
no_stylesheets = True
language = 'pl'
remove_javascript = True
recursions = 0
remove_tags_before = dict(dict(name = 'div', attrs = {'id' : 'print-layer'}))
remove_tags_after = dict(dict(name = 'div', attrs = {'id' : 'print-layer'}))
'''
keep_only_tags =[]
keep_only_tags.append(dict(name = 'table', attrs = {'id' : 'title-table'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'div-header'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'div-content'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'def element-autor'}))
'''
title = u'Wprost'
__author__ = 'matek09'
description = 'Weekly magazine'
encoding = 'ISO-8859-2'
no_stylesheets = True
language = 'pl'
remove_javascript = True
recursions = 0
remove_tags_before = dict(dict(name = 'div', attrs = {'id' : 'print-layer'}))
remove_tags_after = dict(dict(name = 'div', attrs = {'id' : 'print-layer'}))
'''keep_only_tags =[]
keep_only_tags.append(dict(name = 'table', attrs = {'id' : 'title-table'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'div-header'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'div-content'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'def element-autor'}))'''
preprocess_regexps = [(re.compile(r'style="display: none;"'), lambda match: ''),
preprocess_regexps = [(re.compile(r'style="display: none;"'), lambda match: ''),
(re.compile(r'display: block;'), lambda match: ''),
(re.compile(r'\<td\>\<tr\>\<\/table\>'), lambda match: ''),
(re.compile(r'\<table .*?\>'), lambda match: ''),
(re.compile(r'\<tr>'), lambda match: ''),
(re.compile(r'\<td .*?\>'), lambda match: ''),
(re.compile(r'\<div id="footer"\>.*?\</footer\>'), lambda match: '')]
(re.compile(r'\<div id="footer"\>.*?\</footer\>'), lambda match: '')]
remove_tags =[]
remove_tags.append(dict(name = 'div', attrs = {'class' : 'def element-date'}))
remove_tags.append(dict(name = 'div', attrs = {'class' : 'def silver'}))
remove_tags.append(dict(name = 'div', attrs = {'id' : 'content-main-column-right'}))
remove_tags =[]
remove_tags.append(dict(name = 'div', attrs = {'class' : 'def element-date'}))
remove_tags.append(dict(name = 'div', attrs = {'class' : 'def silver'}))
remove_tags.append(dict(name = 'div', attrs = {'id' : 'content-main-column-right'}))
extra_css = '''
.div-header {font-size: x-small; font-weight: bold}
'''
#h2 {font-size: x-large; font-weight: bold}
def is_blocked(self, a):
if a.findNextSibling('img') is None:
return False
else:
return True
extra_css = '''.div-header {font-size: x-small; font-weight: bold}'''
#h2 {font-size: x-large; font-weight: bold}
def is_blocked(self, a):
if a.findNextSibling('img') is None:
return False
else:
return True
def find_last_issue(self):
soup = self.index_to_soup('http://www.wprost.pl/archiwum/')
a = 0
if self.FIND_LAST_FULL_ISSUE:
ico_blocked = soup.findAll('img', attrs={'src' : self.ICO_BLOCKED})
a = ico_blocked[-1].findNext('a', attrs={'title' : re.compile(r'Spis *', re.IGNORECASE | re.DOTALL)})
else:
a = soup.find('a', attrs={'title' : re.compile(r'Spis *', re.IGNORECASE | re.DOTALL)})
self.EDITION = a['href'].replace('/tygodnik/?I=', '')
self.EDITION_SHORT = a['href'].replace('/tygodnik/?I=15', '')
self.cover_url = a.img['src']
def find_last_issue(self):
soup = self.index_to_soup('http://www.wprost.pl/archiwum/')
a = 0
if self.FIND_LAST_FULL_ISSUE:
ico_blocked = soup.findAll('img', attrs={'src' : self.ICO_BLOCKED})
a = ico_blocked[-1].findNext('a', attrs={'title' : re.compile(r'Spis *', re.IGNORECASE | re.DOTALL)})
else:
a = soup.find('a', attrs={'title' : re.compile(r'Spis *', re.IGNORECASE | re.DOTALL)})
self.EDITION = a['href'].replace('/tygodnik/?I=', '')
self.EDITION_SHORT = a['href'].replace('/tygodnik/?I=15', '')
self.cover_url = a.img['src']
def parse_index(self):
self.find_last_issue()
soup = self.index_to_soup('http://www.wprost.pl/tygodnik/?I=' + self.EDITION)
feeds = []
headers = soup.findAll(attrs={'class':'block-header block-header-left mtop20 mbottom20'})
articles_list = soup.findAll(attrs={'class':'standard-box'})
for i in range(len(headers)):
articles = self.find_articles(articles_list[i])
if len(articles) > 0:
section = headers[i].find('a').string
feeds.append((section, articles))
return feeds
def parse_index(self):
self.find_last_issue()
soup = self.index_to_soup('http://www.wprost.pl/tygodnik/?I=' + self.EDITION)
feeds = []
for main_block in soup.findAll(attrs={'id': 'content-main-column-element-content'}):
articles = list(self.find_articles(main_block))
if len(articles) > 0:
section = self.tag_to_string(main_block.find('h3'))
feeds.append((section, articles))
return feeds
def find_articles(self, main_block):
for a in main_block.findAll('a'):
if a.name in "td":
break
if self.EXCLUDE_LOCKED & self.is_blocked(a):
continue
yield {
'title' : self.tag_to_string(a),
'url' : 'http://www.wprost.pl' + a['href'],
'date' : '',
'description' : ''
}
def find_articles(self, main_block):
articles = []
for a in main_block.findAll('a'):
if a.name in "td":
break
if self.EXCLUDE_LOCKED and self.is_blocked(a):
continue
articles.append({
'title' : self.tag_to_string(a),
'url' : 'http://www.wprost.pl' + a['href'],
'date' : '',
'description' : ''
})
return articles

71
recipes/wprost_rss.recipe Normal file
View File

@ -0,0 +1,71 @@
#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2010, matek09, matek09@gmail.com'
__copyright__ = 'Modified 2011, Mariusz Wolek <mariusz_dot_wolek @ gmail dot com>'
__copyright__ = 'Modified 2012, Artur Stachecki <artur.stachecki@gmail.com>'
from calibre.web.feeds.news import BasicNewsRecipe
import re
class Wprost(BasicNewsRecipe):
title = u'Wprost (RSS)'
__author__ = 'matek09'
description = 'Weekly magazine'
encoding = 'ISO-8859-2'
no_stylesheets = True
language = 'pl'
remove_javascript = True
recursions = 0
use_embedded_content = False
remove_empty_feeds = True
remove_tags_before = dict(dict(name = 'div', attrs = {'id' : 'print-layer'}))
remove_tags_after = dict(dict(name = 'div', attrs = {'id' : 'print-layer'}))
'''
keep_only_tags =[]
keep_only_tags.append(dict(name = 'table', attrs = {'id' : 'title-table'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'div-header'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'div-content'}))
keep_only_tags.append(dict(name = 'div', attrs = {'class' : 'def element-autor'}))
'''
preprocess_regexps = [(re.compile(r'style="display: none;"'), lambda match: ''),
(re.compile(r'display: block;'), lambda match: ''),
(re.compile(r'\<td\>\<tr\>\<\/table\>'), lambda match: ''),
(re.compile(r'\<table .*?\>'), lambda match: ''),
(re.compile(r'\<tr>'), lambda match: ''),
(re.compile(r'\<td .*?\>'), lambda match: ''),
(re.compile(r'\<div id="footer"\>.*?\</footer\>'), lambda match: '')]
remove_tags =[]
remove_tags.append(dict(name = 'div', attrs = {'class' : 'def element-date'}))
remove_tags.append(dict(name = 'div', attrs = {'class' : 'def silver'}))
remove_tags.append(dict(name = 'div', attrs = {'id' : 'content-main-column-right'}))
extra_css = '''.div-header {font-size: x-small; font-weight: bold}'''
#h2 {font-size: x-large; font-weight: bold}
feeds = [(u'Tylko u nas', u'http://www.wprost.pl/rss/rss_wprostextra.php'),
(u'Wydarzenia', u'http://www.wprost.pl/rss/rss.php'),
(u'Komentarze', u'http://www.wprost.pl/rss/rss_komentarze.php'),
(u'Wydarzenia: Kraj', u'http://www.wprost.pl/rss/rss_kraj.php'),
(u'Komentarze: Kraj', u'http://www.wprost.pl/rss/rss_komentarze_kraj.php'),
(u'Wydarzenia: Świat', u'http://www.wprost.pl/rss/rss_swiat.php'),
(u'Komentarze: Świat', u'http://www.wprost.pl/rss/rss_komentarze_swiat.php'),
(u'Wydarzenia: Gospodarka', u'http://www.wprost.pl/rss/rss_gospodarka.php'),
(u'Komentarze: Gospodarka', u'http://www.wprost.pl/rss/rss_komentarze_gospodarka.php'),
(u'Wydarzenia: Życie', u'http://www.wprost.pl/rss/rss_zycie.php'),
(u'Komentarze: Życie', u'http://www.wprost.pl/rss/rss_komentarze_zycie.php'),
(u'Wydarzenia: Sport', u'http://www.wprost.pl/rss/rss_sport.php'),
(u'Komentarze: Sport', u'http://www.wprost.pl/rss/rss_komentarze_sport.php'),
(u'Przegląd prasy', u'http://www.wprost.pl/rss/rss_prasa.php')
]
def get_cover_url(self):
soup = self.index_to_soup('http://www.wprost.pl/tygodnik')
cover = soup.find(attrs={'class':'wprost-cover'})
if cover:
self.cover_url = cover['src']
return getattr(self, 'cover_url', self.cover_url)

20
recipes/wsj.recipe
View File

@ -55,20 +55,14 @@ class WallStreetJournal(BasicNewsRecipe):
]
remove_tags_after = [dict(id="article_story_body"), {'class':"article story"},]
use_javascript_to_login = True
def get_browser(self):
br = BasicNewsRecipe.get_browser(self)
if self.username is not None and self.password is not None:
br.open('http://commerce.wsj.com/auth/login')
br.select_form(nr=1)
br['user'] = self.username
br['password'] = self.password
res = br.submit()
raw = res.read()
if 'Welcome,' not in raw and '>Logout<' not in raw and '>Log Out<' not in raw:
raise ValueError('Failed to log in to wsj.com, check your '
'username and password')
return br
def javascript_login(self, br, username, password):
br.visit('https://id.wsj.com/access/pages/wsj/us/login_standalone.html?mg=com-wsj', timeout=120)
f = br.select_form(nr=0)
f['username'] = username
f['password'] = password
br.submit(timeout=120)
def populate_article_metadata(self, article, soup, first):
if first and hasattr(self, 'add_toc_thumbnail'):

2
recipes/zeitde_sub.recipe
View File

@ -88,7 +88,7 @@ class ZeitEPUBAbo(BasicNewsRecipe):
(re.compile(u' \u00AB'), lambda match: u'\u00AB '), # before closing quotation
(re.compile(u'\u00BB '), lambda match: u' \u00BB'), # after opening quotation
# filtering for spaces in large numbers for better readability
(re.compile(r'(?<=\d\d)(?=\d\d\d[ ,\.;\)<\?!-])'), lambda match: u'\u2008'), # end of the number with some character following
(re.compile(r'(?<=\d\d)(?=\d\d\d[ ,;\)<\?!-])'), lambda match: u'\u2008'), # end of the number with some character following
(re.compile(r'(?<=\d\d)(?=\d\d\d. )'), lambda match: u'\u2008'), # end of the number with full-stop following, then space is necessary (avoid file names)
(re.compile(u'(?<=\d)(?=\d\d\d\u2008)'), lambda match: u'\u2008'), # next level
(re.compile(u'(?<=\d)(?=\d\d\d\u2008)'), lambda match: u'\u2008'), # next level

BIN
resources/compiled_coffeescript.zip
View File

Binary file not shown.

8
resources/content_server/browse/browse.css
View File

@ -356,6 +356,10 @@ h2.library_name {
color: red;
}
#booklist a.summary_thumb img {
border: none
}
#booklist > #pagelist { display: none; }
#goto_page_dialog ul {
@ -474,5 +478,9 @@ h2.library_name {
color: red
}
.details a.details_thumb img {
border: none
}
/* }}} */

2
resources/content_server/browse/details.html
View File

@ -1,6 +1,6 @@
<div id="details_{id}" class="details">
<div class="left">
<img alt="Cover of {title}" src="{prefix}/get/cover/{id}" />
<a href="{get_url}" title="Click to read {title} in the {fmt} format" class="details_thumb"><img alt="Cover of {title}" src="{prefix}/get/cover/{id}" /></a>
</div>
<div class="right">
<div class="field formats">{formats}</div>

2
resources/content_server/browse/summary.html
View File

@ -1,6 +1,6 @@
<div id="summary_{id}" class="summary">
<div class="left">
<img alt="Cover of {title}" src="{prefix}/get/thumb_90_120/{id}" />
<a href="{get_url}" class="summary_thumb" title="Click to read {title} in the {fmt} format"><img alt="Cover of {title}" src="{prefix}/get/thumb_90_120/{id}" /></a>
{get_button}
</div>
<div class="right">

2
resources/content_server/index.html
View File

@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" version="XHTML 1.1" xml:lang="en">
<head>
<title>calibre library</title>

12
resources/default_tweaks.py
View File

@ -464,12 +464,15 @@ server_listen_on = '0.0.0.0'
# on at your own risk!
unified_title_toolbar_on_osx = False
#: Save original file when converting from same format to same format
#: Save original file when converting/polishing from same format to same format
# When calibre does a conversion from the same format to the same format, for
# example, from EPUB to EPUB, the original file is saved, so that in case the
# conversion is poor, you can tweak the settings and run it again. By setting
# this to False you can prevent calibre from saving the original file.
# Similarly, by setting save_original_format_when_polishing to False you can
# prevent calibre from saving the original file when polishing.
save_original_format = True
save_original_format_when_polishing = True
#: Number of recently viewed books to show
# Right-clicking the View button shows a list of recently viewed books. Control
@ -514,3 +517,10 @@ default_tweak_format = None
# your library and your personal editing style.
preselect_first_completion = False
#: Recognize numbers inside text when sorting
# This means that when sorting on text fields like title the text "Book 2"
# will sort before the text "Book 100". If you want this behavior, set
# numeric_collation = True note that doing so will cause problems with text
# that starts with numbers and is a little slower.
numeric_collation = False

BIN
resources/images/icon_choose.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

BIN
resources/images/polish.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

4
setup/check.py
View File

@ -95,9 +95,9 @@ class Check(Command):
errors = True
self.report_errors(w)
else:
from calibre.utils.serve_coffee import check_coffeescript
try:
subprocess.check_call(['coffee', '-c', '-p', f],
stdout=open(os.devnull, 'wb'))
check_coffeescript(f)
except:
errors = True
if errors:

100
setup/iso_639/ca.po
View File

@ -12,14 +12,14 @@ msgstr ""
"Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
"devel@lists.alioth.debian.org>\n"
"POT-Creation-Date: 2011-11-25 14:01+0000\n"
"PO-Revision-Date: 2013-01-19 20:28+0000\n"
"PO-Revision-Date: 2013-02-19 18:01+0000\n"
"Last-Translator: Ferran Rius <frius64@hotmail.com>\n"
"Language-Team: Catalan <linux@softcatala.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Launchpad-Export-Date: 2013-01-20 04:36+0000\n"
"X-Generator: Launchpad (build 16430)\n"
"X-Launchpad-Export-Date: 2013-02-20 04:50+0000\n"
"X-Generator: Launchpad (build 16491)\n"
"Language: ca\n"
#. name for aaa
@ -1920,7 +1920,7 @@ msgstr "Arára; Mato Grosso"
#. name for axk
msgid "Yaka (Central African Republic)"
msgstr "Yaka (República Centreafricana)"
msgstr "Yaka (República Centrafricana)"
#. name for axm
msgid "Armenian; Middle"
@ -3528,7 +3528,7 @@ msgstr "Buamu"
#. name for boy
msgid "Bodo (Central African Republic)"
msgstr "Bodo (República Centreafricana)"
msgstr "Bodo (República Centrafricana)"
#. name for boz
msgid "Bozo; Tiéyaxo"
@ -7928,7 +7928,7 @@ msgstr "Oromo; occidental"
#. name for gba
msgid "Gbaya (Central African Republic)"
msgstr "Gbaya (República Centreafricana)"
msgstr "Gbaya (República Centrafricana)"
#. name for gbb
msgid "Kaytetye"
@ -11184,7 +11184,7 @@ msgstr ""
#. name for kbn
msgid "Kare (Central African Republic)"
msgstr "Kare (República Centreafricana)"
msgstr "Kare (República Centrafricana)"
#. name for kbo
msgid "Keliko"
@ -20720,7 +20720,7 @@ msgstr "Pitjantjatjara"
#. name for pka
msgid "Prākrit; Ardhamāgadhī"
msgstr ""
msgstr "Pràcrit; Ardhamagadhi"
#. name for pkb
msgid "Pokomo"
@ -20776,31 +20776,31 @@ msgstr "Polonombauk"
#. name for plc
msgid "Palawano; Central"
msgstr ""
msgstr "Palawà; Central"
#. name for pld
msgid "Polari"
msgstr ""
msgstr "Polari"
#. name for ple
msgid "Palu'e"
msgstr ""
msgstr "Palue"
#. name for plg
msgid "Pilagá"
msgstr ""
msgstr "Pilagà"
#. name for plh
msgid "Paulohi"
msgstr ""
msgstr "Paulohi"
#. name for pli
msgid "Pali"
msgstr ""
msgstr "Pali"
#. name for plj
msgid "Polci"
msgstr ""
msgstr "Polci"
#. name for plk
msgid "Shina; Kohistani"
@ -20812,19 +20812,19 @@ msgstr "Palaung; Shwe"
#. name for pln
msgid "Palenquero"
msgstr ""
msgstr "Palenquero"
#. name for plo
msgid "Popoluca; Oluta"
msgstr ""
msgstr "Popoluca; Oluta"
#. name for plp
msgid "Palpa"
msgstr ""
msgstr "Palpa"
#. name for plq
msgid "Palaic"
msgstr ""
msgstr "Palaic"
#. name for plr
msgid "Senoufo; Palaka"
@ -20840,15 +20840,15 @@ msgstr "Malgaix; Plateau"
#. name for plu
msgid "Palikúr"
msgstr ""
msgstr "Palikur"
#. name for plv
msgid "Palawano; Southwest"
msgstr ""
msgstr "Palawà; Sudoccidental"
#. name for plw
msgid "Palawano; Brooke's Point"
msgstr ""
msgstr "Palawà; Brooke"
#. name for ply
msgid "Bolyu"
@ -20856,43 +20856,43 @@ msgstr ""
#. name for plz
msgid "Paluan"
msgstr ""
msgstr "Paluà"
#. name for pma
msgid "Paama"
msgstr ""
msgstr "Paama"
#. name for pmb
msgid "Pambia"
msgstr ""
msgstr "Pambia"
#. name for pmc
msgid "Palumata"
msgstr ""
msgstr "Palumata"
#. name for pme
msgid "Pwaamei"
msgstr ""
msgstr "Pwaamei"
#. name for pmf
msgid "Pamona"
msgstr ""
msgstr "Pamona"
#. name for pmh
msgid "Prākrit; Māhārāṣṭri"
msgstr ""
msgstr "Pràcrit; Maharastri"
#. name for pmi
msgid "Pumi; Northern"
msgstr ""
msgstr "Pumi; Septentrional"
#. name for pmj
msgid "Pumi; Southern"
msgstr ""
msgstr "Pumi; Meridional"
#. name for pmk
msgid "Pamlico"
msgstr ""
msgstr "Algonquí Carolina"
#. name for pml
msgid "Lingua Franca"
@ -20904,11 +20904,11 @@ msgstr "Pol"
#. name for pmn
msgid "Pam"
msgstr ""
msgstr "Pam"
#. name for pmo
msgid "Pom"
msgstr ""
msgstr "Pom"
#. name for pmq
msgid "Pame; Northern"
@ -20916,11 +20916,11 @@ msgstr "Pame; Septentrional"
#. name for pmr
msgid "Paynamar"
msgstr ""
msgstr "Paynamar"
#. name for pms
msgid "Piemontese"
msgstr ""
msgstr "Piemontès"
#. name for pmt
msgid "Tuamotuan"
@ -20956,7 +20956,7 @@ msgstr "Panjabi; Occidental"
#. name for pnc
msgid "Pannei"
msgstr ""
msgstr "Pannei"
#. name for pne
msgid "Penan; Western"
@ -20964,11 +20964,11 @@ msgstr "Penan; Occidental"
#. name for png
msgid "Pongu"
msgstr ""
msgstr "Pongu"
#. name for pnh
msgid "Penrhyn"
msgstr ""
msgstr "Penrhyn"
#. name for pni
msgid "Aoheng"
@ -20976,27 +20976,27 @@ msgstr ""
#. name for pnm
msgid "Punan Batu 1"
msgstr ""
msgstr "Punan Batu"
#. name for pnn
msgid "Pinai-Hagahai"
msgstr ""
msgstr "Pinai-Hagahai"
#. name for pno
msgid "Panobo"
msgstr ""
msgstr "Panobo"
#. name for pnp
msgid "Pancana"
msgstr ""
msgstr "Pancana"
#. name for pnq
msgid "Pana (Burkina Faso)"
msgstr ""
msgstr "Pana (Burkina Faso)"
#. name for pnr
msgid "Panim"
msgstr ""
msgstr "Panim"
#. name for pns
msgid "Ponosakan"
@ -21028,7 +21028,7 @@ msgstr ""
#. name for pnz
msgid "Pana (Central African Republic)"
msgstr ""
msgstr "Pana (República Centrafricana)"
#. name for poc
msgid "Poqomam"
@ -21056,7 +21056,7 @@ msgstr ""
#. name for poi
msgid "Popoluca; Highland"
msgstr ""
msgstr "Popoluca; Muntanya"
#. name for pok
msgid "Pokangá"
@ -21084,7 +21084,7 @@ msgstr ""
#. name for poq
msgid "Popoluca; Texistepec"
msgstr ""
msgstr "Popoluca; Texistepec"
#. name for por
msgid "Portuguese"
@ -21092,7 +21092,7 @@ msgstr "Portuguès"
#. name for pos
msgid "Popoluca; Sayula"
msgstr ""
msgstr "Popoluca; Sayula"
#. name for pot
msgid "Potawatomi"
@ -21336,7 +21336,7 @@ msgstr "Paixtú; Central"
#. name for psu
msgid "Prākrit; Sauraseni"
msgstr ""
msgstr "Pràcrit; Sauraseni"
#. name for psw
msgid "Port Sandwich"

94
setup/iso_639/nl.po
View File

@ -12,14 +12,14 @@ msgstr ""
"Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
"devel@lists.alioth.debian.org>\n"
"POT-Creation-Date: 2011-11-25 14:01+0000\n"
"PO-Revision-Date: 2012-10-24 18:16+0000\n"
"PO-Revision-Date: 2013-02-04 07:01+0000\n"
"Last-Translator: drMerry <Unknown>\n"
"Language-Team: Dutch <vertaling@vrijschrift.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Launchpad-Export-Date: 2012-10-25 05:35+0000\n"
"X-Generator: Launchpad (build 16179)\n"
"X-Launchpad-Export-Date: 2013-02-05 04:44+0000\n"
"X-Generator: Launchpad (build 16468)\n"
"Language: nl\n"
#. name for aaa
@ -728,7 +728,7 @@ msgstr "Aimol"
#. name for ain
msgid "Ainu (Japan)"
msgstr ""
msgstr "Ainu (Japan)"
#. name for aio
msgid "Aiton"
@ -2496,7 +2496,7 @@ msgstr "Wit-Russisch; Belarussisch"
#. name for bem
msgid "Bemba (Zambia)"
msgstr ""
msgstr "Bemba (Zambia)"
#. name for ben
msgid "Bengali"
@ -2892,7 +2892,7 @@ msgstr "Bimoba"
#. name for bin
msgid "Bini"
msgstr "Bini"
msgstr "Bini; Edo"
#. name for bio
msgid "Nai"
@ -4684,7 +4684,7 @@ msgstr ""
#. name for car
msgid "Carib; Galibi"
msgstr ""
msgstr "Caribische talen"
#. name for cas
msgid "Tsimané"
@ -6360,7 +6360,7 @@ msgstr ""
#. name for div
msgid "Dhivehi"
msgstr ""
msgstr "Divehi"
#. name for diw
msgid "Dinka; Northwestern"
@ -6604,7 +6604,7 @@ msgstr "Vietnamese dong"
#. name for doi
msgid "Dogri (macrolanguage)"
msgstr ""
msgstr "Dogri"
#. name for dok
msgid "Dondo"
@ -7508,7 +7508,7 @@ msgstr ""
#. name for fan
msgid "Fang (Equatorial Guinea)"
msgstr ""
msgstr "Fang"
#. name for fao
msgid "Faroese"
@ -7584,7 +7584,7 @@ msgstr "Fijisch"
#. name for fil
msgid "Filipino"
msgstr ""
msgstr "Filipijns"
#. name for fin
msgid "Finnish"
@ -8192,7 +8192,7 @@ msgstr ""
#. name for gez
msgid "Geez"
msgstr "Geez"
msgstr "Ge'ez"
#. name for gfk
msgid "Patpatar"
@ -9016,7 +9016,7 @@ msgstr ""
#. name for gwi
msgid "Gwichʼin"
msgstr ""
msgstr "Gwichʼin"
#. name for gwj
msgid "/Gwi"
@ -10076,7 +10076,7 @@ msgstr ""
#. name for iii
msgid "Yi; Sichuan"
msgstr ""
msgstr "Yi; Sichuan - Nuosu"
#. name for ijc
msgid "Izon"
@ -11076,11 +11076,11 @@ msgstr ""
#. name for kal
msgid "Kalaallisut"
msgstr ""
msgstr "Groenlands"
#. name for kam
msgid "Kamba (Kenya)"
msgstr ""
msgstr "Kamba (Kenya)"
#. name for kan
msgid "Kannada"
@ -11784,7 +11784,7 @@ msgstr ""
#. name for khm
msgid "Khmer; Central"
msgstr ""
msgstr "Khmer, Cambodjaans"
#. name for khn
msgid "Khandesi"
@ -12488,7 +12488,7 @@ msgstr ""
#. name for kok
msgid "Konkani (macrolanguage)"
msgstr ""
msgstr "Konkani"
#. name for kol
msgid "Kol (Papua New Guinea)"
@ -12500,7 +12500,7 @@ msgstr "Komi"
#. name for kon
msgid "Kongo"
msgstr "Kongo"
msgstr "Kikongo"
#. name for koo
msgid "Konzo"
@ -13736,7 +13736,7 @@ msgstr ""
#. name for lao
msgid "Lao"
msgstr "Lao"
msgstr "Laotiaans"
#. name for lap
msgid "Laka (Chad)"
@ -14872,7 +14872,7 @@ msgstr ""
#. name for lug
msgid "Ganda"
msgstr "Ganda"
msgstr "Luganda"
#. name for lui
msgid "Luiseno"
@ -15468,7 +15468,7 @@ msgstr ""
#. name for men
msgid "Mende (Sierra Leone)"
msgstr ""
msgstr "Mende"
#. name for meo
msgid "Malay; Kedah"
@ -15832,7 +15832,7 @@ msgstr ""
#. name for mic
msgid "Mi'kmaq"
msgstr ""
msgstr "Mi'kmaq; Micmac"
#. name for mid
msgid "Mandaic"
@ -16812,7 +16812,7 @@ msgstr ""
#. name for msa
msgid "Malay (macrolanguage)"
msgstr ""
msgstr "Maleis"
#. name for msb
msgid "Masbatenyo"
@ -17680,7 +17680,7 @@ msgstr "Nauruaans"
#. name for nav
msgid "Navajo"
msgstr ""
msgstr "Navajo"
#. name for naw
msgid "Nawuri"
@ -18068,7 +18068,7 @@ msgstr ""
#. name for new
msgid "Bhasa; Nepal"
msgstr ""
msgstr "Newari; Nepal"
#. name for nex
msgid "Neme"
@ -18296,7 +18296,7 @@ msgstr ""
#. name for nia
msgid "Nias"
msgstr "Niaas"
msgstr "Nias"
#. name for nib
msgid "Nakame"
@ -18972,7 +18972,7 @@ msgstr ""
#. name for nqo
msgid "N'Ko"
msgstr "Nko"
msgstr "N'Ko"
#. name for nra
msgid "Ngom"
@ -19080,7 +19080,7 @@ msgstr ""
#. name for nso
msgid "Sotho; Northern"
msgstr ""
msgstr "Pedi; Sepedi; Noord-Sothotisch"
#. name for nsp
msgid "Nepalese Sign Language"
@ -19296,7 +19296,7 @@ msgstr ""
#. name for nwc
msgid "Newari; Old"
msgstr ""
msgstr "Newari; Klassiek Nepal"
#. name for nwe
msgid "Ngwe"
@ -19372,7 +19372,7 @@ msgstr ""
#. name for nya
msgid "Nyanja"
msgstr ""
msgstr "Nyanja"
#. name for nyb
msgid "Nyangbo"
@ -20028,7 +20028,7 @@ msgstr "Spaans; oud"
#. name for oss
msgid "Ossetian"
msgstr ""
msgstr "Ossetisch"
#. name for ost
msgid "Osatu"
@ -20212,7 +20212,7 @@ msgstr "Pampanga"
#. name for pan
msgid "Panjabi"
msgstr ""
msgstr "Punjabi"
#. name for pao
msgid "Paiute; Northern"
@ -21848,7 +21848,7 @@ msgstr ""
#. name for rar
msgid "Maori; Cook Islands"
msgstr ""
msgstr "Rarotongan; Cookeilanden Maori"
#. name for ras
msgid "Tegali"
@ -22220,7 +22220,7 @@ msgstr ""
#. name for rom
msgid "Romany"
msgstr "Roma"
msgstr "Romani"
#. name for ron
msgid "Romanian"
@ -23380,7 +23380,7 @@ msgstr ""
#. name for sma
msgid "Sami; Southern"
msgstr ""
msgstr "Samisch; zuid, Laps; zuid"
#. name for smb
msgid "Simbari"
@ -23396,7 +23396,7 @@ msgstr ""
#. name for sme
msgid "Sami; Northern"
msgstr ""
msgstr "Samisch; noord, Laps; noord"
#. name for smf
msgid "Auwe"
@ -23428,7 +23428,7 @@ msgstr ""
#. name for smn
msgid "Sami; Inari"
msgstr ""
msgstr "Sami; Inari, Laps; Inari"
#. name for smo
msgid "Samoan"
@ -23448,7 +23448,7 @@ msgstr ""
#. name for sms
msgid "Sami; Skolt"
msgstr ""
msgstr "Sami; Skolt, Laps; Skolt"
#. name for smt
msgid "Simte"
@ -24188,7 +24188,7 @@ msgstr ""
#. name for swa
msgid "Swahili (macrolanguage)"
msgstr ""
msgstr "Swahili"
#. name for swb
msgid "Comorian; Maore"
@ -24344,7 +24344,7 @@ msgstr ""
#. name for syc
msgid "Syriac; Classical"
msgstr ""
msgstr "Syriac; Klassiek"
#. name for syi
msgid "Seki"
@ -25332,7 +25332,7 @@ msgstr ""
#. name for tlh
msgid "Klingon"
msgstr ""
msgstr "Klingon; tlhIngan-Hol"
#. name for tli
msgid "Tlingit"
@ -27412,7 +27412,7 @@ msgstr ""
#. name for wal
msgid "Wolaytta"
msgstr ""
msgstr "Walamo"
#. name for wam
msgid "Wampanoag"
@ -27436,7 +27436,7 @@ msgstr ""
#. name for war
msgid "Waray (Philippines)"
msgstr ""
msgstr "Waray (Filipijns)"
#. name for was
msgid "Washo"
@ -28240,7 +28240,7 @@ msgstr ""
#. name for xal
msgid "Kalmyk"
msgstr ""
msgstr "Kalmyk"
#. name for xam
msgid "/Xam"
@ -30144,7 +30144,7 @@ msgstr ""
#. name for zab
msgid "Zapotec; San Juan Guelavía"
msgstr ""
msgstr "Zapotec"
#. name for zac
msgid "Zapotec; Ocotlán"
@ -30308,7 +30308,7 @@ msgstr ""
#. name for zha
msgid "Zhuang"
msgstr ""
msgstr "Zhuang, Tsjoeang"
#. name for zhb
msgid "Zhaba"

16
setup/iso_639/pt.po
View File

@ -10,19 +10,19 @@ msgstr ""
"Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
"devel@lists.alioth.debian.org>\n"
"POT-Creation-Date: 2011-11-25 14:01+0000\n"
"PO-Revision-Date: 2011-09-27 16:52+0000\n"
"Last-Translator: Kovid Goyal <Unknown>\n"
"PO-Revision-Date: 2013-02-18 02:41+0000\n"
"Last-Translator: pedro jorge oliveira <pedrojorgeoliveira93@gmail.com>\n"
"Language-Team: Portuguese <pt@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Launchpad-Export-Date: 2011-11-26 05:34+0000\n"
"X-Generator: Launchpad (build 14381)\n"
"X-Launchpad-Export-Date: 2013-02-19 04:56+0000\n"
"X-Generator: Launchpad (build 16491)\n"
"Language: pt\n"
#. name for aaa
msgid "Ghotuo"
msgstr ""
msgstr "Ghotuo"
#. name for aab
msgid "Alumu-Tesu"
@ -498,7 +498,7 @@ msgstr ""
#. name for afr
msgid "Afrikaans"
msgstr "Africanos"
msgstr "Africano"
#. name for afs
msgid "Creole; Afro-Seminole"
@ -910,7 +910,7 @@ msgstr ""
#. name for ale
msgid "Aleut"
msgstr "aleúte"
msgstr "Aleúte"
#. name for alf
msgid "Alege"
@ -30818,7 +30818,7 @@ msgstr ""
#. name for zxx
msgid "No linguistic content"
msgstr ""
msgstr "Sem conteúdo linguistico"
#. name for zyb
msgid "Zhuang; Yongbei"

10
setup/iso_639/pt_BR.po
View File

@ -9,14 +9,14 @@ msgstr ""
"Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
"devel@lists.alioth.debian.org>\n"
"POT-Creation-Date: 2011-11-25 14:01+0000\n"
"PO-Revision-Date: 2012-12-21 03:31+0000\n"
"Last-Translator: Fábio Malcher Miranda <mirand863@hotmail.com>\n"
"PO-Revision-Date: 2013-02-17 21:57+0000\n"
"Last-Translator: Neliton Pereira Jr. <nelitonpjr@gmail.com>\n"
"Language-Team: Brazilian Portuguese\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Launchpad-Export-Date: 2012-12-22 04:59+0000\n"
"X-Generator: Launchpad (build 16378)\n"
"X-Launchpad-Export-Date: 2013-02-18 04:49+0000\n"
"X-Generator: Launchpad (build 16491)\n"
"Language: \n"
#. name for aaa
@ -141,7 +141,7 @@ msgstr ""
#. name for abh
msgid "Arabic; Tajiki"
msgstr ""
msgstr "Arábico; Tajiki"
#. name for abi
msgid "Abidji"

28
setup/iso_639/vi.po
View File

@ -9,43 +9,43 @@ msgstr ""
"Report-Msgid-Bugs-To: Debian iso-codes team <pkg-isocodes-"
"devel@lists.alioth.debian.org>\n"
"POT-Creation-Date: 2011-11-25 14:01+0000\n"
"PO-Revision-Date: 2011-09-27 16:56+0000\n"
"Last-Translator: Clytie Siddall <clytie@riverland.net.au>\n"
"PO-Revision-Date: 2013-02-15 06:39+0000\n"
"Last-Translator: baduong <Unknown>\n"
"Language-Team: Vietnamese <gnomevi-list@lists.sourceforge.net>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Launchpad-Export-Date: 2011-11-26 05:44+0000\n"
"X-Generator: Launchpad (build 14381)\n"
"X-Launchpad-Export-Date: 2013-02-16 04:56+0000\n"
"X-Generator: Launchpad (build 16491)\n"
"Language: vi\n"
#. name for aaa
msgid "Ghotuo"
msgstr ""
msgstr "Ghotuo"
#. name for aab
msgid "Alumu-Tesu"
msgstr ""
msgstr "Alumu-Tesu"
#. name for aac
msgid "Ari"
msgstr ""
msgstr "Ari"
#. name for aad
msgid "Amal"
msgstr ""
msgstr "Amal"
#. name for aae
msgid "Albanian; Arbëreshë"
msgstr ""
msgstr "An-ba-ni"
#. name for aaf
msgid "Aranadan"
msgstr ""
msgstr "Aranadan"
#. name for aag
msgid "Ambrak"
msgstr ""
msgstr "Ambrak"
#. name for aah
msgid "Arapesh; Abu'"
@ -30817,7 +30817,7 @@ msgstr ""
#. name for zxx
msgid "No linguistic content"
msgstr ""
msgstr "Không có nội dung kiểu ngôn ngữ"
#. name for zyb
msgid "Zhuang; Yongbei"
@ -30829,11 +30829,11 @@ msgstr ""
#. name for zyj
msgid "Zhuang; Youjiang"
msgstr ""
msgstr "Zhuang; Youjiang"
#. name for zyn
msgid "Zhuang; Yongnan"
msgstr ""
msgstr "Zhuang; Yongnan"
#. name for zyp
msgid "Zyphe"

2
setup/resources.py
View File

@ -26,7 +26,7 @@ def get_opts_from_parser(parser):
class Coffee(Command): # {{{
description = 'Compile coffeescript files into javascript'
COFFEE_DIRS = ('ebooks/oeb/display',)
COFFEE_DIRS = ('ebooks/oeb/display', 'ebooks/oeb/polish')
def add_options(self, parser):
parser.add_option('--watch', '-w', action='store_true', default=False,

2
src/calibre/constants.py
View File

@ -4,7 +4,7 @@ __license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'
__appname__ = u'calibre'
numeric_version = (0, 9, 16)
numeric_version = (0, 9, 20)
__version__ = u'.'.join(map(unicode, numeric_version))
__author__ = u"Kovid Goyal <kovid@kovidgoyal.net>"

2
src/calibre/customize/__init__.py
View File

@ -712,7 +712,7 @@ class ViewerPlugin(Plugin): # {{{
def run_javascript(self, evaljs):
'''
This method is called every time a document has finished laoding. Use
This method is called every time a document has finished loading. Use
it in the same way as load_javascript().
'''
pass

16
src/calibre/customize/builtins.py
View File

@ -624,12 +624,6 @@ from calibre.library.catalogs.epub_mobi import EPUB_MOBI
plugins += [CSV_XML, BIBTEX, EPUB_MOBI]
# }}}
# EPUB Fix plugins {{{
from calibre.ebooks.epub.fix.unmanifested import Unmanifested
from calibre.ebooks.epub.fix.epubcheck import Epubcheck
plugins += [Unmanifested, Epubcheck]
# }}}
# Profiles {{{
from calibre.customize.profiles import input_profiles, output_profiles
plugins += input_profiles + output_profiles
@ -757,13 +751,14 @@ plugins += [
# New metadata download plugins {{{
from calibre.ebooks.metadata.sources.google import GoogleBooks
from calibre.ebooks.metadata.sources.amazon import Amazon
from calibre.ebooks.metadata.sources.edelweiss import Edelweiss
from calibre.ebooks.metadata.sources.openlibrary import OpenLibrary
from calibre.ebooks.metadata.sources.isbndb import ISBNDB
from calibre.ebooks.metadata.sources.overdrive import OverDrive
from calibre.ebooks.metadata.sources.douban import Douban
from calibre.ebooks.metadata.sources.ozon import Ozon
plugins += [GoogleBooks, Amazon, OpenLibrary, ISBNDB, OverDrive, Douban, Ozon]
plugins += [GoogleBooks, Amazon, Edelweiss, OpenLibrary, ISBNDB, OverDrive, Douban, Ozon]
# }}}
@ -789,6 +784,11 @@ class ActionConvert(InterfaceActionBase):
actual_plugin = 'calibre.gui2.actions.convert:ConvertAction'
description = _('Convert books to various ebook formats')
class ActionPolish(InterfaceActionBase):
name = 'Polish Books'
actual_plugin = 'calibre.gui2.actions.polish:PolishAction'
description = _('Fine tune your ebooks')
class ActionDelete(InterfaceActionBase):
name = 'Remove Books'
actual_plugin = 'calibre.gui2.actions.delete:DeleteAction'
@ -924,7 +924,7 @@ class ActionPluginUpdater(InterfaceActionBase):
plugins += [ActionAdd, ActionFetchAnnotations, ActionGenerateCatalog,
ActionConvert, ActionDelete, ActionEditMetadata, ActionView,
ActionFetchNews, ActionSaveToDisk, ActionQuickview,
ActionFetchNews, ActionSaveToDisk, ActionQuickview, ActionPolish,
ActionShowBookDetails,ActionRestart, ActionOpenFolder, ActionConnectShare,
ActionSendToDevice, ActionHelp, ActionPreferences, ActionSimilarBooks,
ActionAddToLibrary, ActionEditCollections, ActionChooseLibrary,

9
src/calibre/customize/profiles.py
View File

@ -452,6 +452,13 @@ class SamsungGalaxy(TabletOutput):
'a resolution of 600x1280')
screen_size = comic_screen_size = (600, 1280)
class NookHD(TabletOutput):
name = 'Nook HD+'
short_name = 'nook_hd_plus'
description = _('Intended for the Nook HD+ and similar tablet devices with '
'a resolution of 1280x1920')
screen_size = comic_screen_size = (1280, 1920)
class SonyReaderOutput(OutputProfile):
name = 'Sony Reader'
@ -786,7 +793,7 @@ output_profiles = [OutputProfile, SonyReaderOutput, SonyReader300Output,
SonyReader900Output, MSReaderOutput, MobipocketOutput, HanlinV3Output,
HanlinV5Output, CybookG3Output, CybookOpusOutput, KindleOutput,
iPadOutput, iPad3Output, KoboReaderOutput, TabletOutput, SamsungGalaxy,
SonyReaderLandscapeOutput, KindleDXOutput, IlliadOutput,
SonyReaderLandscapeOutput, KindleDXOutput, IlliadOutput, NookHD,
IRexDR1000Output, IRexDR800Output, JetBook5Output, NookOutput,
BambookOutput, NookColorOutput, PocketBook900Output, PocketBookPro912Output,
GenericEink, GenericEinkLarge, KindleFireOutput, KindlePaperWhiteOutput]

12
src/calibre/customize/ui.py
View File

@ -17,7 +17,6 @@ from calibre.devices.interface import DevicePlugin
from calibre.ebooks.metadata import MetaInformation
from calibre.utils.config import (make_config_dir, Config, ConfigProxy,
plugin_dir, OptionParser)
from calibre.ebooks.epub.fix import ePubFixer
from calibre.ebooks.metadata.sources.base import Source
from calibre.constants import DEBUG
@ -92,7 +91,7 @@ def restore_plugin_state_to_default(plugin_or_name):
config['enabled_plugins'] = ep
default_disabled_plugins = set([
'Overdrive', 'Douban Books', 'OZON.ru',
'Overdrive', 'Douban Books', 'OZON.ru', 'Edelweiss',
])
def is_disabled(plugin):
@ -489,15 +488,6 @@ def disabled_device_plugins():
yield plugin
# }}}
# epub fixers {{{
def epub_fixers():
for plugin in _initialized_plugins:
if isinstance(plugin, ePubFixer):
if not is_disabled(plugin):
if platform in plugin.supported_platforms:
yield plugin
# }}}
# Metadata sources2 {{{
def metadata_plugins(capabilities):
capabilities = frozenset(capabilities)

101
src/calibre/db/backend.py
View File

@ -16,15 +16,14 @@ import apsw
from calibre import isbytestring, force_unicode, prints
from calibre.constants import (iswindows, filesystem_encoding,
preferred_encoding)
from calibre.ptempfile import PersistentTemporaryFile, SpooledTemporaryFile
from calibre.db import SPOOL_SIZE
from calibre.ptempfile import PersistentTemporaryFile
from calibre.db.schema_upgrades import SchemaUpgrade
from calibre.library.field_metadata import FieldMetadata
from calibre.ebooks.metadata import title_sort, author_to_author_sort
from calibre.utils.icu import strcmp
from calibre.utils.icu import sort_key
from calibre.utils.config import to_json, from_json, prefs, tweaks
from calibre.utils.date import utcfromtimestamp, parse_date
from calibre.utils.filenames import is_case_sensitive
from calibre.utils.filenames import (is_case_sensitive, samefile, hardlink_file)
from calibre.db.tables import (OneToOneTable, ManyToOneTable, ManyToManyTable,
SizeTable, FormatsTable, AuthorsTable, IdentifiersTable,
CompositeTable, LanguagesTable)
@ -173,7 +172,9 @@ def _author_to_author_sort(x):
return author_to_author_sort(x.replace('|', ','))
def icu_collator(s1, s2):
return strcmp(force_unicode(s1, 'utf-8'), force_unicode(s2, 'utf-8'))
return cmp(sort_key(force_unicode(s1, 'utf-8')),
sort_key(force_unicode(s2, 'utf-8')))
# }}}
# Unused aggregators {{{
@ -400,6 +401,7 @@ class DB(object):
defs['gui_restriction'] = defs['cs_restriction'] = ''
defs['categories_using_hierarchy'] = []
defs['column_color_rules'] = []
defs['column_icon_rules'] = []
defs['grouped_search_make_user_categories'] = []
defs['similar_authors_search_key'] = 'authors'
defs['similar_authors_match_kind'] = 'match_any'
@ -854,38 +856,75 @@ class DB(object):
ans = {}
if path is not None:
stat = os.stat(path)
ans['path'] = path
ans['size'] = stat.st_size
ans['mtime'] = utcfromtimestamp(stat.st_mtime)
return ans
def cover(self, path, as_file=False, as_image=False,
as_path=False):
def has_format(self, book_id, fmt, fname, path):
return self.format_abspath(book_id, fmt, fname, path) is not None
def copy_cover_to(self, path, dest, windows_atomic_move=None, use_hardlink=False):
path = os.path.join(self.library_path, path, 'cover.jpg')
ret = None
if os.access(path, os.R_OK):
try:
if windows_atomic_move is not None:
if not isinstance(dest, basestring):
raise Exception("Error, you must pass the dest as a path when"
" using windows_atomic_move")
if os.access(path, os.R_OK) and dest and not samefile(dest, path):
windows_atomic_move.copy_path_to(path, dest)
return True
else:
if os.access(path, os.R_OK):
try:
f = lopen(path, 'rb')
except (IOError, OSError):
time.sleep(0.2)
f = lopen(path, 'rb')
except (IOError, OSError):
time.sleep(0.2)
f = lopen(path, 'rb')
with f:
if as_path:
pt = PersistentTemporaryFile('_dbcover.jpg')
with pt:
shutil.copyfileobj(f, pt)
return pt.name
if as_file:
ret = SpooledTemporaryFile(SPOOL_SIZE)
shutil.copyfileobj(f, ret)
ret.seek(0)
else:
ret = f.read()
if as_image:
from PyQt4.Qt import QImage
i = QImage()
i.loadFromData(ret)
ret = i
return ret
with f:
if hasattr(dest, 'write'):
shutil.copyfileobj(f, dest)
if hasattr(dest, 'flush'):
dest.flush()
return True
elif dest and not samefile(dest, path):
if use_hardlink:
try:
hardlink_file(path, dest)
return True
except:
pass
with lopen(dest, 'wb') as d:
shutil.copyfileobj(f, d)
return True
return False
def copy_format_to(self, book_id, fmt, fname, path, dest,
windows_atomic_move=None, use_hardlink=False):
path = self.format_abspath(book_id, fmt, fname, path)
if path is None:
return False
if windows_atomic_move is not None:
if not isinstance(dest, basestring):
raise Exception("Error, you must pass the dest as a path when"
" using windows_atomic_move")
if dest and not samefile(dest, path):
windows_atomic_move.copy_path_to(path, dest)
else:
if hasattr(dest, 'write'):
with lopen(path, 'rb') as f:
shutil.copyfileobj(f, dest)
if hasattr(dest, 'flush'):
dest.flush()
elif dest and not samefile(dest, path):
if use_hardlink:
try:
hardlink_file(path, dest)
return True
except:
pass
with lopen(path, 'rb') as f, lopen(dest, 'wb') as d:
shutil.copyfileobj(f, d)
return True
# }}}

217
src/calibre/db/cache.py
View File

@ -8,16 +8,21 @@ __copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import os, traceback
from io import BytesIO
from collections import defaultdict
from functools import wraps, partial
from calibre.db import SPOOL_SIZE
from calibre.db.categories import get_categories
from calibre.db.locking import create_locks, RecordLock
from calibre.db.errors import NoSuchFormat
from calibre.db.fields import create_field
from calibre.db.search import Search
from calibre.db.tables import VirtualTable
from calibre.db.lazy import FormatMetadata, FormatsList
from calibre.ebooks.metadata.book.base import Metadata
from calibre.ptempfile import (base_dir, PersistentTemporaryFile,
SpooledTemporaryFile)
from calibre.utils.date import now
from calibre.utils.icu import sort_key
@ -103,27 +108,6 @@ class Cache(object):
def field_metadata(self):
return self.backend.field_metadata
def _format_abspath(self, book_id, fmt):
'''
Return absolute path to the ebook file of format `format`
WARNING: This method will return a dummy path for a network backend DB,
so do not rely on it, use format(..., as_path=True) instead.
Currently used only in calibredb list, the viewer and the catalogs (via
get_data_as_dict()).
Apart from the viewer, I don't believe any of the others do any file
I/O with the results of this call.
'''
try:
name = self.fields['formats'].format_fname(book_id, fmt)
path = self._field_for('path', book_id).replace('/', os.sep)
except:
return None
if name and path:
return self.backend.format_abspath(book_id, fmt, name, path)
def _get_metadata(self, book_id, get_user_categories=True): # {{{
mi = Metadata(None, template_cache=self.formatter_template_cache)
author_ids = self._field_ids_for('authors', book_id)
@ -162,7 +146,7 @@ class Cache(object):
if not formats:
good_formats = None
else:
mi.format_metadata = FormatMetadata(self, id, formats)
mi.format_metadata = FormatMetadata(self, book_id, formats)
good_formats = FormatsList(formats, mi.format_metadata)
mi.formats = good_formats
mi.has_cover = _('Yes') if self._field_for('cover', book_id,
@ -227,6 +211,12 @@ class Cache(object):
self.fields['ondevice'] = create_field('ondevice',
VirtualTable('ondevice'))
for name, field in self.fields.iteritems():
if name[0] == '#' and name.endswith('_index'):
field.series_field = self.fields[name[:-len('_index')]]
elif name == 'series_index':
field.series_field = self.fields['series']
@read_api
def field_for(self, name, book_id, default_value=None):
'''
@ -397,15 +387,184 @@ class Cache(object):
:param as_path: If True return the image as a path pointing to a
temporary file
'''
if as_file:
ret = SpooledTemporaryFile(SPOOL_SIZE)
if not self.copy_cover_to(book_id, ret): return
ret.seek(0)
elif as_path:
pt = PersistentTemporaryFile('_dbcover.jpg')
with pt:
if not self.copy_cover_to(book_id, pt): return
ret = pt.name
else:
buf = BytesIO()
if not self.copy_cover_to(book_id, buf): return
ret = buf.getvalue()
if as_image:
from PyQt4.Qt import QImage
i = QImage()
i.loadFromData(ret)
ret = i
return ret
@api
def copy_cover_to(self, book_id, dest, use_hardlink=False):
'''
Copy the cover to the file like object ``dest``. Returns False
if no cover exists or dest is the same file as the current cover.
dest can also be a path in which case the cover is
copied to it iff the path is different from the current path (taking
case sensitivity into account).
'''
with self.read_lock:
try:
path = self._field_for('path', book_id).replace('/', os.sep)
except:
return None
return False
with self.record_lock.lock(book_id):
return self.backend.cover(path, as_file=as_file, as_image=as_image,
as_path=as_path)
return self.backend.copy_cover_to(path, dest,
use_hardlink=use_hardlink)
@api
def copy_format_to(self, book_id, fmt, dest, use_hardlink=False):
'''
Copy the format ``fmt`` to the file like object ``dest``. If the
specified format does not exist, raises :class:`NoSuchFormat` error.
dest can also be a path, in which case the format is copied to it, iff
the path is different from the current path (taking case sensitivity
into account).
'''
with self.read_lock:
try:
name = self.fields['formats'].format_fname(book_id, fmt)
path = self._field_for('path', book_id).replace('/', os.sep)
except:
raise NoSuchFormat('Record %d has no %s file'%(book_id, fmt))
with self.record_lock.lock(book_id):
return self.backend.copy_format_to(book_id, fmt, name, path, dest,
use_hardlink=use_hardlink)
@read_api
def format_abspath(self, book_id, fmt):
'''
Return absolute path to the ebook file of format `format`
Currently used only in calibredb list, the viewer and the catalogs (via
get_data_as_dict()).
Apart from the viewer, I don't believe any of the others do any file
I/O with the results of this call.
'''
try:
name = self.fields['formats'].format_fname(book_id, fmt)
path = self._field_for('path', book_id).replace('/', os.sep)
except:
return None
if name and path:
return self.backend.format_abspath(book_id, fmt, name, path)
@read_api
def has_format(self, book_id, fmt):
'Return True iff the format exists on disk'
try:
name = self.fields['formats'].format_fname(book_id, fmt)
path = self._field_for('path', book_id).replace('/', os.sep)
except:
return False
return self.backend.has_format(book_id, fmt, name, path)
@read_api
def formats(self, book_id, verify_formats=True):
'''
Return tuple of all formats for the specified book. If verify_formats
is True, verifies that the files exist on disk.
'''
ans = self.field_for('formats', book_id)
if verify_formats and ans:
try:
path = self._field_for('path', book_id).replace('/', os.sep)
except:
return ()
def verify(fmt):
try:
name = self.fields['formats'].format_fname(book_id, fmt)
except:
return False
return self.backend.has_format(book_id, fmt, name, path)
ans = tuple(x for x in ans if verify(x))
return ans
@api
def format(self, book_id, fmt, as_file=False, as_path=False, preserve_filename=False):
'''
Return the ebook format as a bytestring or `None` if the format doesn't exist,
or we don't have permission to write to the ebook file.
:param as_file: If True the ebook format is returned as a file object. Note
that the file object is a SpooledTemporaryFile, so if what you want to
do is copy the format to another file, use :method:`copy_format_to`
instead for performance.
:param as_path: Copies the format file to a temp file and returns the
path to the temp file
:param preserve_filename: If True and returning a path the filename is
the same as that used in the library. Note that using
this means that repeated calls yield the same
temp file (which is re-created each time)
'''
with self.read_lock:
ext = ('.'+fmt.lower()) if fmt else ''
try:
fname = self.fields['formats'].format_fname(book_id, fmt)
except:
return None
fname += ext
if as_path:
if preserve_filename:
bd = base_dir()
d = os.path.join(bd, 'format_abspath')
try:
os.makedirs(d)
except:
pass
ret = os.path.join(d, fname)
with self.record_lock.lock(book_id):
try:
self.copy_format_to(book_id, fmt, ret)
except NoSuchFormat:
return None
else:
with PersistentTemporaryFile(ext) as pt, self.record_lock.lock(book_id):
try:
self.copy_format_to(book_id, fmt, pt)
except NoSuchFormat:
return None
ret = pt.name
elif as_file:
ret = SpooledTemporaryFile(SPOOL_SIZE)
with self.record_lock.lock(book_id):
try:
self.copy_format_to(book_id, fmt, ret)
except NoSuchFormat:
return None
ret.seek(0)
# Various bits of code try to use the name as the default
# title when reading metadata, so set it
ret.name = fname
else:
buf = BytesIO()
with self.record_lock.lock(book_id):
try:
self.copy_format_to(book_id, fmt, buf)
except NoSuchFormat:
return None
ret = buf.getvalue()
return ret
@read_api
def multisort(self, fields, ids_to_sort=None):
@ -455,6 +614,14 @@ class Cache(object):
return get_categories(self, sort=sort, book_ids=book_ids,
icon_map=icon_map)
@write_api
def set_field(self, name, book_id_to_val_map, allow_case_change=True):
# TODO: Specialize title/authors to also update path
# TODO: Handle updating caches used by composite fields
dirtied = self.fields[name].writer.set_books(
book_id_to_val_map, self.backend, allow_case_change=allow_case_change)
return dirtied
# }}}
class SortKey(object):

22
src/calibre/db/categories.py
View File

@ -12,6 +12,7 @@ from functools import partial
from operator import attrgetter
from future_builtins import map
from calibre.ebooks.metadata import author_to_author_sort
from calibre.library.field_metadata import TagsIcons
from calibre.utils.config_base import tweaks
from calibre.utils.icu import sort_key
@ -35,6 +36,8 @@ class Tag(object):
self.avg_rating = avg/2.0 if avg is not None else 0
self.sort = sort
self.use_sort_as_name = use_sort_as_name
if tooltip is None:
tooltip = '(%s:%s)'%(category, name)
if self.avg_rating > 0:
if tooltip:
tooltip = tooltip + ': '
@ -64,8 +67,8 @@ def find_categories(field_metadata):
def create_tag_class(category, fm, icon_map):
cat = fm[category]
dt = cat['datatype']
icon = None
tooltip = None if category in {'formats', 'identifiers'} else ('(' + category + ')')
label = fm.key_to_label(category)
if icon_map:
if not fm.is_custom_field(category):
@ -75,20 +78,19 @@ def create_tag_class(category, fm, icon_map):
icon = icon_map['custom:']
icon_map[category] = icon
is_editable = category not in {'news', 'rating', 'languages', 'formats',
'identifiers'}
'identifiers'} and dt != 'composite'
if (tweaks['categories_use_field_for_author_name'] == 'author_sort' and
(category == 'authors' or
(cat['display'].get('is_names', False) and
cat['is_custom'] and cat['is_multiple'] and
cat['datatype'] == 'text'))):
dt == 'text'))):
use_sort_as_name = True
else:
use_sort_as_name = False
return partial(Tag, use_sort_as_name=use_sort_as_name, icon=icon,
tooltip=tooltip, is_editable=is_editable,
category=category)
is_editable=is_editable, category=category)
def clean_user_categories(dbcache):
user_cats = dbcache.pref('user_categories', {})
@ -148,8 +150,16 @@ def get_categories(dbcache, sort='name', book_ids=None, icon_map=None):
elif category == 'news':
cats = dbcache.fields['tags'].get_news_category(tag_class, book_ids)
else:
cat = fm[category]
brm = book_rating_map
if cat['datatype'] == 'rating' and category != 'rating':
brm = dbcache.fields[category].book_value_map
cats = dbcache.fields[category].get_categories(
tag_class, book_rating_map, lang_map, book_ids)
tag_class, brm, lang_map, book_ids)
if (category != 'authors' and cat['datatype'] == 'text' and
cat['is_multiple'] and cat['display'].get('is_names', False)):
for item in cats:
item.sort = author_to_author_sort(item.sort)
sort_categories(cats, sort)
categories[category] = cats

3
src/calibre/db/fields.py
View File

@ -12,6 +12,7 @@ from threading import Lock
from collections import defaultdict, Counter
from calibre.db.tables import ONE_ONE, MANY_ONE, MANY_MANY
from calibre.db.write import Writer
from calibre.ebooks.metadata import title_sort
from calibre.utils.config_base import tweaks
from calibre.utils.icu import sort_key
@ -44,6 +45,8 @@ class Field(object):
self.category_formatter = lambda x:'\u2605'*int(x/2)
elif name == 'languages':
self.category_formatter = calibre_langcode_to_name
self.writer = Writer(self)
self.series_field = None
@property
def metadata(self):

51
src/calibre/db/tests/base.py
View File

@ -7,19 +7,36 @@ __license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import unittest, os, shutil
import unittest, os, shutil, tempfile, atexit
from functools import partial
from io import BytesIO
from future_builtins import map
rmtree = partial(shutil.rmtree, ignore_errors=True)
class BaseTest(unittest.TestCase):
def setUp(self):
self.library_path = self.mkdtemp()
self.create_db(self.library_path)
def tearDown(self):
shutil.rmtree(self.library_path)
def create_db(self, library_path):
from calibre.library.database2 import LibraryDatabase2
if LibraryDatabase2.exists_at(library_path):
raise ValueError('A library already exists at %r'%library_path)
src = os.path.join(os.path.dirname(__file__), 'metadata.db')
db = os.path.join(library_path, 'metadata.db')
shutil.copyfile(src, db)
return db
dest = os.path.join(library_path, 'metadata.db')
shutil.copyfile(src, dest)
db = LibraryDatabase2(library_path)
db.set_cover(1, I('lt.png', data=True))
db.set_cover(2, I('polish.png', data=True))
db.add_format(1, 'FMT1', BytesIO(b'book1fmt1'), index_is_id=True)
db.add_format(1, 'FMT2', BytesIO(b'book1fmt2'), index_is_id=True)
db.add_format(2, 'FMT1', BytesIO(b'book2fmt1'), index_is_id=True)
return dest
def init_cache(self, library_path):
from calibre.db.backend import DB
@ -29,20 +46,38 @@ class BaseTest(unittest.TestCase):
cache.init()
return cache
def mkdtemp(self):
ans = tempfile.mkdtemp(prefix='db_test_')
atexit.register(rmtree, ans)
return ans
def init_old(self, library_path):
from calibre.library.database2 import LibraryDatabase2
return LibraryDatabase2(library_path)
def clone_library(self, library_path):
if not hasattr(self, 'clone_dir'):
self.clone_dir = tempfile.mkdtemp()
atexit.register(rmtree, self.clone_dir)
self.clone_count = 0
self.clone_count += 1
dest = os.path.join(self.clone_dir, str(self.clone_count))
shutil.copytree(library_path, dest)
return dest
def compare_metadata(self, mi1, mi2):
allfk1 = mi1.all_field_keys()
allfk2 = mi2.all_field_keys()
self.assertEqual(allfk1, allfk2)
all_keys = {'format_metadata', 'id', 'application_id',
'author_sort_map', 'author_link_map', 'book_size',
'ondevice_col', 'last_modified'}.union(allfk1)
'author_sort_map', 'author_link_map', 'book_size',
'ondevice_col', 'last_modified', 'has_cover',
'cover_data'}.union(allfk1)
for attr in all_keys:
if attr == 'user_metadata': continue
if attr == 'format_metadata': continue # TODO: Not implemented yet
attr1, attr2 = getattr(mi1, attr), getattr(mi2, attr)
if attr == 'formats':
continue # TODO: Not implemented yet
attr1, attr2 = map(lambda x:tuple(x) if x else (), (attr1, attr2))
self.assertEqual(attr1, attr2,
'%s not the same: %r != %r'%(attr, attr1, attr2))

125
src/calibre/db/tests/reading.py
View File

@ -7,21 +7,13 @@ __license__ = 'GPL v3'
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import shutil, unittest, tempfile, datetime
from cStringIO import StringIO
import unittest, datetime
from calibre.utils.date import utc_tz
from calibre.db.tests.base import BaseTest
class ReadingTest(BaseTest):
def setUp(self):
self.library_path = tempfile.mkdtemp()
self.create_db(self.library_path)
def tearDown(self):
shutil.rmtree(self.library_path)
def test_read(self): # {{{
'Test the reading of data from the database'
cache = self.init_cache(self.library_path)
@ -55,7 +47,7 @@ class ReadingTest(BaseTest):
'#tags':(),
'#yesno':None,
'#comments': None,
'size':None,
},
2 : {
@ -66,7 +58,7 @@ class ReadingTest(BaseTest):
'series' : 'A Series One',
'series_index': 1.0,
'tags':('Tag One', 'Tag Two'),
'formats': (),
'formats': ('FMT1',),
'rating': 4.0,
'identifiers': {'test':'one'},
'timestamp': datetime.datetime(2011, 9, 5, 21, 6,
@ -86,6 +78,7 @@ class ReadingTest(BaseTest):
'#tags':('My Tag One', 'My Tag Two'),
'#yesno':True,
'#comments': '<div>My Comments One<p></p></div>',
'size':9,
},
1 : {
'title': 'Title Two',
@ -96,7 +89,7 @@ class ReadingTest(BaseTest):
'series_index': 2.0,
'rating': 6.0,
'tags': ('Tag One', 'News'),
'formats':(),
'formats':('FMT1', 'FMT2'),
'identifiers': {'test':'two'},
'timestamp': datetime.datetime(2011, 9, 6, 6, 0,
tzinfo=utc_tz),
@ -115,6 +108,7 @@ class ReadingTest(BaseTest):
'#tags':('My Tag Two',),
'#yesno':False,
'#comments': '<div>My Comments Two<p></p></div>',
'size':9,
},
}
@ -172,22 +166,41 @@ class ReadingTest(BaseTest):
'Test get_metadata() returns the same data for both backends'
from calibre.library.database2 import LibraryDatabase2
old = LibraryDatabase2(self.library_path)
for i in xrange(1, 3):
old.add_format(i, 'txt%d'%i, StringIO(b'random%d'%i),
index_is_id=True)
old.add_format(i, 'text%d'%i, StringIO(b'random%d'%i),
index_is_id=True)
old_metadata = {i:old.get_metadata(i, index_is_id=True) for i in
old_metadata = {i:old.get_metadata(
i, index_is_id=True, get_cover=True, cover_as_data=True) for i in
xrange(1, 4)}
for mi in old_metadata.itervalues():
mi.format_metadata = dict(mi.format_metadata)
if mi.formats:
mi.formats = tuple(mi.formats)
old = None
cache = self.init_cache(self.library_path)
new_metadata = {i:cache.get_metadata(i) for i in xrange(1, 4)}
new_metadata = {i:cache.get_metadata(
i, get_cover=True, cover_as_data=True) for i in xrange(1, 4)}
cache = None
for mi2, mi1 in zip(new_metadata.values(), old_metadata.values()):
self.compare_metadata(mi1, mi2)
# }}}
def test_get_cover(self): # {{{
'Test cover() returns the same data for both backends'
from calibre.library.database2 import LibraryDatabase2
old = LibraryDatabase2(self.library_path)
covers = {i: old.cover(i, index_is_id=True) for i in old.all_ids()}
old = None
cache = self.init_cache(self.library_path)
for book_id, cdata in covers.iteritems():
self.assertEqual(cdata, cache.cover(book_id), 'Reading of cover failed')
f = cache.cover(book_id, as_file=True)
self.assertEqual(cdata, f.read() if f else f, 'Reading of cover as file failed')
if cdata:
with open(cache.cover(book_id, as_path=True), 'rb') as f:
self.assertEqual(cdata, f.read(), 'Reading of cover as path failed')
else:
self.assertEqual(cdata, cache.cover(book_id, as_path=True),
'Reading of null cover as path failed')
# }}}
@ -227,8 +240,12 @@ class ReadingTest(BaseTest):
# User categories
'@Good Authors:One', '@Good Series.good tags:two',
# TODO: Tests for searching the size and #formats columns and
# cover:true|false
# Cover/Formats
'cover:true', 'cover:false', 'formats:true', 'formats:false',
'formats:#>1', 'formats:#=1', 'formats:=fmt1', 'formats:=fmt2',
'formats:=fmt1 or formats:fmt2', '#formats:true', '#formats:false',
'#formats:fmt1', '#formats:fmt2', '#formats:fmt1 and #formats:fmt2',
)}
old = None
@ -247,9 +264,67 @@ class ReadingTest(BaseTest):
old = LibraryDatabase2(self.library_path)
old_categories = old.get_categories()
cache = self.init_cache(self.library_path)
import pprint
pprint.pprint(old_categories)
pprint.pprint(cache.get_categories())
new_categories = cache.get_categories()
self.assertEqual(set(old_categories), set(new_categories),
'The set of old categories is not the same as the set of new categories')
def compare_category(category, old, new):
for attr in ('name', 'original_name', 'id', 'count',
'is_hierarchical', 'is_editable', 'is_searchable',
'id_set', 'avg_rating', 'sort', 'use_sort_as_name',
'tooltip', 'icon', 'category'):
oval, nval = getattr(old, attr), getattr(new, attr)
if (
(category in {'rating', '#rating'} and attr in {'id_set', 'sort'}) or
(category == 'series' and attr == 'sort') or # Sorting is wrong in old
(category == 'identifiers' and attr == 'id_set') or
(category == '@Good Series') or # Sorting is wrong in old
(category == 'news' and attr in {'count', 'id_set'}) or
(category == 'formats' and attr == 'id_set')
):
continue
self.assertEqual(oval, nval,
'The attribute %s for %s in category %s does not match. Old is %r, New is %r'
%(attr, old.name, category, oval, nval))
for category in old_categories:
old, new = old_categories[category], new_categories[category]
self.assertEqual(len(old), len(new),
'The number of items in the category %s is not the same'%category)
for o, n in zip(old, new):
compare_category(category, o, n)
# }}}
def test_get_formats(self): # {{{
'Test reading ebook formats using the format() method'
from calibre.library.database2 import LibraryDatabase2
old = LibraryDatabase2(self.library_path)
ids = old.all_ids()
lf = {i:set(old.formats(i, index_is_id=True).split(',')) if old.formats(
i, index_is_id=True) else set() for i in ids}
formats = {i:{f:old.format(i, f, index_is_id=True) for f in fmts} for
i, fmts in lf.iteritems()}
old = None
cache = self.init_cache(self.library_path)
for book_id, fmts in lf.iteritems():
self.assertEqual(fmts, set(cache.formats(book_id)),
'Set of formats is not the same')
for fmt in fmts:
old = formats[book_id][fmt]
self.assertEqual(old, cache.format(book_id, fmt),
'Old and new format disagree')
f = cache.format(book_id, fmt, as_file=True)
self.assertEqual(old, f.read(),
'Failed to read format as file')
with open(cache.format(book_id, fmt, as_path=True,
preserve_filename=True), 'rb') as f:
self.assertEqual(old, f.read(),
'Failed to read format as path')
with open(cache.format(book_id, fmt, as_path=True), 'rb') as f:
self.assertEqual(old, f.read(),
'Failed to read format as path')
# }}}

127
src/calibre/db/tests/writing.py Normal file
View File

@ -0,0 +1,127 @@
#!/usr/bin/env python
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
from __future__ import (unicode_literals, division, absolute_import,
print_function)
__license__ = 'GPL v3'
__copyright__ = '2013, Kovid Goyal <kovid at kovidgoyal.net>'
__docformat__ = 'restructuredtext en'
import unittest
from collections import namedtuple
from functools import partial
from calibre.utils.date import UNDEFINED_DATE
from calibre.db.tests.base import BaseTest
class WritingTest(BaseTest):
@property
def cloned_library(self):
return self.clone_library(self.library_path)
def create_getter(self, name, getter=None):
if getter is None:
if name.endswith('_index'):
ans = lambda db:partial(db.get_custom_extra, index_is_id=True,
label=name[1:].replace('_index', ''))
else:
ans = lambda db:partial(db.get_custom, label=name[1:],
index_is_id=True)
else:
ans = lambda db:partial(getattr(db, getter), index_is_id=True)
return ans
def create_setter(self, name, setter=None):
if setter is None:
ans = lambda db:partial(db.set_custom, label=name[1:], commit=True)
else:
ans = lambda db:partial(getattr(db, setter), commit=True)
return ans
def create_test(self, name, vals, getter=None, setter=None ):
T = namedtuple('Test', 'name vals getter setter')
return T(name, vals, self.create_getter(name, getter),
self.create_setter(name, setter))
def run_tests(self, tests):
results = {}
for test in tests:
results[test] = []
for val in test.vals:
cl = self.cloned_library
cache = self.init_cache(cl)
cache.set_field(test.name, {1: val})
cached_res = cache.field_for(test.name, 1)
del cache
db = self.init_old(cl)
getter = test.getter(db)
sqlite_res = getter(1)
if test.name.endswith('_index'):
val = float(val) if val is not None else 1.0
self.assertEqual(sqlite_res, val,
'Failed setting for %s with value %r, sqlite value not the same. val: %r != sqlite_val: %r'%(
test.name, val, val, sqlite_res))
else:
test.setter(db)(1, val)
old_cached_res = getter(1)
self.assertEqual(old_cached_res, cached_res,
'Failed setting for %s with value %r, cached value not the same. Old: %r != New: %r'%(
test.name, val, old_cached_res, cached_res))
db.refresh()
old_sqlite_res = getter(1)
self.assertEqual(old_sqlite_res, sqlite_res,
'Failed setting for %s, sqlite value not the same: %r != %r'%(
test.name, old_sqlite_res, sqlite_res))
del db
def test_one_one(self):
'Test setting of values in one-one fields'
tests = [self.create_test('#yesno', (True, False, 'true', 'false', None))]
for name, getter, setter in (
('#series_index', None, None),
('series_index', 'series_index', 'set_series_index'),
('#float', None, None),
):
vals = ['1.5', None, 0, 1.0]
tests.append(self.create_test(name, tuple(vals), getter, setter))
for name, getter, setter in (
('pubdate', 'pubdate', 'set_pubdate'),
('timestamp', 'timestamp', 'set_timestamp'),
('#date', None, None),
):
tests.append(self.create_test(
name, ('2011-1-12', UNDEFINED_DATE, None), getter, setter))
for name, getter, setter in (
('title', 'title', 'set_title'),
('uuid', 'uuid', 'set_uuid'),
('author_sort', 'author_sort', 'set_author_sort'),
('sort', 'title_sort', 'set_title_sort'),
('#comments', None, None),
('comments', 'comments', 'set_comment'),
):
vals = ['something', None]
if name not in {'comments', '#comments'}:
# Setting text column to '' returns None in the new backend
# and '' in the old. I think None is more correct.
vals.append('')
if name == 'comments':
# Again new behavior of deleting comment rather than setting
# empty string is more correct.
vals.remove(None)
tests.append(self.create_test(name, tuple(vals), getter, setter))
self.run_tests(tests)
def tests():
return unittest.TestLoader().loadTestsFromTestCase(WritingTest)
def run():
unittest.TextTestRunner(verbosity=2).run(tests())
if __name__ == '__main__':
run()

Some files were not shown because too many files have changed in this diff Show More