Update ad filter

Recent changes to ads in search results caused Whoogle to display ads
for certain searches. In particular, ads recently started appearing
grouped into one div, as opposed to a singular ad per div. This was
accompanied by the div label "ads" (instead of just "ad"), which threw
off the existing ad filter. The ad keyword blacklist has been updated
accordingly, and has been enhanced to only check against alpha chars for
each label.

This only seems to have affected English language searches, and only for
very specific searches.
このコミットが含まれているのは:
Ben Busby 2022-02-25 23:02:58 -07:00
コミット b28fa86e33
この署名に対応する既知のキーがデータベースに存在しません
GPGキーID: B9B7231E01D924A1
1個のファイルの変更7行の追加5行の削除

ファイルの表示

@ -18,10 +18,11 @@ BLANK_B64 = ('data:image/png;base64,'
# Ad keywords
BLACKLIST = [
'ad', 'anuncio', 'annuncio', 'annonce', 'Anzeige', '广告', '廣告', 'Reklama',
'Реклама', 'Anunț', '광고', 'annons', 'Annonse', 'Iklan', '広告', 'Augl.',
'Mainos', 'Advertentie', 'إعلان', 'Գովազդ', 'विज्ञापन', 'Reklam', 'آگهی',
'Reklāma', 'Reklaam', 'Διαφήμιση', 'מודעה', 'Hirdetés', 'Anúncio'
'ad', 'ads', 'anuncio', 'annuncio', 'annonce', 'Anzeige', '广告', '廣告',
'Reklama', 'Реклама', 'Anunț', '광고', 'annons', 'Annonse', 'Iklan',
'広告', 'Augl.', 'Mainos', 'Advertentie', 'إعلان', 'Գովազդ', 'विज्ञापन',
'Reklam', 'آگهی', 'Reklāma', 'Reklaam', 'Διαφήμιση', 'מודעה', 'Hirdetés',
'Anúncio'
]
SITE_ALTS = {
@ -89,7 +90,8 @@ def has_ad_content(element: str) -> bool:
bool: True/False for the element containing an ad
"""
return (element.upper() in (value.upper() for value in BLACKLIST)
element_str = ''.join(filter(str.isalpha, element))
return (element_str.upper() in (value.upper() for value in BLACKLIST)
or '' in element)