- Removed hardcoded User Agent strings and replaced them with a fallback mechanism using DEFAULT_FALLBACK_UA.
- Updated gen_user_agent function to ensure compatibility with older configurations.
- Bumped version to 1.1.1 to reflect changes in User Agent management.
- Revised README to reflect changes in Google search behavior and Whoogle's response strategies.
- Implemented a User Agent pool for improved request handling, including fallback mechanisms.
- Added configuration options for displaying the User Agent in search results.
- Introduced a command-line tool for generating custom User Agent strings.
- Enhanced request headers to include additional parameters for better compatibility with Google services.
* Improving readability, removing unnecessary elements, etc.
* Minor changes to comment style and favicon response code
---------
Co-authored-by: Ben Busby <contact@benbusby.com>
Including the version portion of the URL now redirects to search results
for the name of the bang file, rather than returning the bang file
itself. Removing the version from the URL returns the correct bang file.
- WHOOGLE_SHOW_FAVICONS: Default on, can be set to 0 to hide favicons
and skip the request for fetching them
- WHOOGLE_UPDATE_CHECK: Default on, can be set to 0 to disable the
daily check for new versions released on github
Closes#1098Closes#1059
This should fix the annoyance with browsers like Firefox not caching
POST request responses. By redirecting a POST search to be a GET request
instead (with an encrypted query string), the page can be cached and
successfully navigated back to after visiting a result.
DDG provides favicons using the url format
icons.duckduckgo.com/ip2/{site}.ico
This can be used to fetch favicons in the event that the default
"/favicon.ico" path does not work.
Audio controls are now always shown by default (mostly found in searches
that contain word pronunciation guides).
Site icons were moved to the left side of the results.
When starting whoogle from another directory, the path to the calculator
widget was previously invalid. It now specifies the path relative to the widget
loader file.
The calculator was previously triggered for partial matches with words
like "calc", which meant searches containing the word "calcium" would
cause the calculator widget to appear.
Redirects to alternative frontends can now be defined using the
WHOOGLE_REDIRECTS environment variable. Usage is documented in the
readme, but is basically defined as <parent>:<new>.
Closes#988
Defines separate environment variables for setting mobile vs desktop user
agents
Defines an environment variable for using the client's User-Agent
Co-authored-by: Ben Busby <contact@benbusby.com>
More MarkupResemblesLocatorWarning warnings have been appearing. This
seems to be caused by parsing HTML content that contains a URL.
This new change suppresses the warning at the root level of the app
before any content has been parsed, so this error shouldn't appear
again.
Fixes#968
An invalid parsing warning was being thrown by the latest version of the
bs4 library. This suppresses that warning from being shown in the
console.
A 404 handler was added to move logging from the console to the error
template, since a lot of users assumed that 404 errors from the result
page were problems with Whoogle itself.
Fixes#967
This adds a simple calculator widget, somewhat similar to the one presented
when searching calculator on Google.
Also, it adds somewhat of a template for making the addition of new widgets
easier via the app/utils/widgets.py file. My eventual plan is to use this to
create more widgets that appear in Google, such as a color picker, timer, etc.
---------
Co-authored-by: Ben Busby <contact@benbusby.com>
Medium redirects needed further cleanup to account for instances where a
link contains a subdomain that would not make sense in a Farside
redirect link.
Fixes#947
The url prefix was not included when reconstructing the root url using
X-Forwarded-* headers, causing some elements to fail to load properly.
Fixes#937
Add a function to check if target_word contains CJK characters
If a search term contains Chinese, Japanese, or Korean characters,
the term is bolded in search results regardless of whitespace.
CJK characters: Chinese, Japanese (hiragana, katakana, kanji),
and Korean (hangul syllables, hangul jamo)
Co-authored-by: Ben Busby <contact@benbusby.com>
Moved the cleaner functions to app/utils/escaper.py
Removed unused import 're'
Moved the cleaner functionalities to the "search.py" and "routes.py"
Making sure escaped chars stay escaped during process
Replaced "<" and ">" with "andlt;" and "andgt;", respectively. This way,
when the 'response' object get loaded to bsoup (which happens several times
throughout the process between search.py and routes.py), bsoup will not
unescape them.
Parent sites using a 'www' subdomain or something similar were not
redirecting properly. This updates the hostname check to only validate
against the primary domain, except for Wikipedia since the subdomain is
used for interface translation in that case.
Fixes#901
Replacing result links and text when site alts are enabled is now part
of its own function, and handles replacement of link location and link
description separately.
Fixes#880
This can be updated later to allow users with cookies enabled to use a
key that is unique to their session (if they want, not mandatory), but
for now it makes more sense to just use a single key for all queries
from all users. This should eliminate a lot of issues that users have
reported where they are unable to decrypt queries or page elements due
to an expired/renewed session key.
New changes to google search now include ads prefixed with the keyword
"sponsored". This update should remove these from appearing in search
results.
Fixes#871
Don-Swanson
rstefko
Ben Busby
Althior
Andrew
Akopov
Matt Burns
David Shen
Gautam Korlam
MoistCat
João
elliot
Ahmad Alkadri
Charles Zawacki
Xabi