StoreID Support Caching Youtube dan

Facebook HTTPS

Facebook

Twitter

Google+

Ini catatan pribadi, isinya storeid untuk squid-3.x Support caching youtube (thanks Ces Pun),
Facebook HTTPS, serta beberapa web lainnya (playstore, reverbnation, dan sebagainya). Untuk
mengaktifkan log storeid, silakan set $debug=1; pada storeid, dan buat file /tmp/storeid.log yang
sudah di chown ke user proxy/squid.
Untuk mengecek log, silakan tail -f /tmp/storeid.log.
Note : untuk konfigurasi membutuhkan squid-3.4.7 yang sudah di patch untuk bug # 3937 (storeid
loop).
Konfigurasi
# storeid *test*

acl urlrewrite dstdomain .fbcdn.net .akamaihd.net

acl speedtest url_regex -i speedtest\/.*\.(jpg|txt)\?.*

acl reverbnation url_regex -i reverbnation.*audio_player.*ec_stream_song.*$

acl utmgif url_regex -i utm.gif.*

acl playstoreandroid url_regex -i

c.android.clients.google.com.market.GetBinary.GetBinary.*

acl idyoutube url_regex -i

youtube.*(ptracking|stream_204|player_204).*(v\=|docid\=|video_id\=).*$

acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?

acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?

acl loop_302 http_status 302

acl getmethod method GET

store_id_program /usr/bin/perl /etc/squid/storeid

store_id_children 20 startup=0 idle=1 concurrency=5

store_id_access deny !getmethod

store_id_access allow urlrewrite

store_id_access allow speedtest

store_id_access allow reverbnation

store_id_access allow utmgif

store_id_access allow playstoreandroid

store_id_access allow idyoutube

store_id_access allow videoyoutube

store_id_access deny all

store_miss deny videoyoutube loop_302

send_hit deny videoyoutube loop_302

refresh_pattern (akamaihd|fbcdn)\.net 14400 99% 518400 ignore-no-store ignoreprivate ignore-reload ignore-must-revalidate store-stale

refresh_pattern -i squid\.internal 14400 99% 518400

ignore-reload ignore-must-revalidate store-stale

ignore-no-store ignore-private

refresh_pattern \.(jpg|png|gif|css|ico)($|\?) 14400 99% 518400 ignore-no-store

ignore-private reload-into-ims ignore-must-revalidate store-stale

refresh_pattern . 0 99% 518400

store-stale

ignore-no-store ignore-private reload-into-ims

Isi storeid.pl
#!/usr/bin/perl

# storeid.pl with debug opt - based on storeurl.pl

# @ http://www2.fh-lausitz.de/launic/comp/misc/squid/projekt_youtube/

# mods by cespun and fajar @ ipfire.id

use IO::File;

$|=1;

STDOUT->autoflush(1);

$debug=0;

## recommended:0

$bypassallrules=0;

## recommended:0

$sucks="";

## unused

$sucks="sucks" if ($debug>=1);

$timenow="";

$printtimenow=1;

my $logfile = '/tmp/storeid.log';

## print timenow: 0|1

open my $logfh, '>>', $logfile

or die "Couldn't open $logfile for appending: $!\n" if $debug;

$logfh->autoflush(1) if $debug;

while (<>) {

$timenow=time()." " if ($printtimenow);

print $logfh "$timenow"."in : $_" if ($debug>=1);

chop;

my $myURL = $_;

@X = split(" ",$myURL);

$a = $X[0]; ## channel id

$b = $X[1]; ## url

$c = $X[2]; ## ip address

$u = $b; ## url

if ($bypassallrules){

$out="$u"; ## map 1:1

} elsif ($u=~
m/http.*\.(fbcdn|akamaihd)\.net\/h(profile|photos).*[\d\w].*\/([\w]\d+x\d+\/.*\.[\d
\w]{3}).*/) {

$out="OK store-id=http://fbcdn.net.squid.internal/" . $2 . "/" . $3 ;

} elsif ($u=~
m/^http(.*)static(.*)(akamaihd|fbcdn).net\/rsrc.php\/(.*\/.*\/(.*).(js|css|png|gif)
)(\?(.*)|$)/) {

$out="OK store-id=http://fbcdn.net.squid.internal/static/" . $5 . "." . $6

;

} elsif ($u=~ m/^https?\:\/\/.*utm.gif.*/) {

$out="OK store-id=http://google-analytics.squid.internal/__utm.gif";

} elsif ($u=~ m/^https?\:\/\/.*\/speedtest\/(.*\.(jpg|txt)).*/) {

$out="OK store-id=http://speedtest.squid.internal/" . $1;

} elsif ($u=~ m/^https?\:\/\/.*\/(.*\..*(mp4|3gp|flv))\?.*/) {

$out="OK store-id=http://video-file.squid.internal/" . $1;

} elsif ($u=~
m/^https?\:\/\/c2lo\.reverbnation\.com\/audio_player\/ec_stream_song\/(.*)\?.*/) {

$out="OK store-id=http://reverbnation.squid.internal/" . $1;

} elsif ($u=~
m/^https?\:\/\/.*\.c\.android\.clients\.google\.com\/market\/GetBinary\/GetBinary\/
(.*\/.*)\?.*/) {

$out="OK store-id=http://playstore-android.squid.internal/" . $1;

} elsif ($u=~ m/^https?\:\/\/.*youtube.*ptracking.*/){

@video_id = m/[&?]video_id\=([^\&\s]*)/;

@cpn = m/[&?]cpn\=([^\&\s]*)/;

unless (-e "/tmp/@cpn"){

open FILE, ">/tmp/@cpn";

print FILE "@video_id";

close FILE;

$out="ERR";

} elsif ($u=~ m/^https?\:\/\/.*youtube.*stream_204.*/){

@docid = m/[&?]docid\=([^\&\s]*)/;

@cpn = m/[&?]cpn\=([^\&\s]*)/;

unless (-e "/tmp/@cpn"){

open FILE, ">/tmp/@cpn";

print FILE "@docid";

close FILE;

$out="ERR";

} elsif ($u=~ m/^https?\:\/\/.*youtube.*player_204.*/){

@v = m/[&?]v\=([^\&\s]*)/;

@cpn = m/[&?]cpn\=([^\&\s]*)/;

unless (-e "/tmp/@cpn"){

open FILE, ">/tmp/@cpn";

print FILE "@v";

close FILE;

$out="ERR";

} elsif ($u=~ m/^https?\:\/\/.*(youtube|googlevideo).*videoplayback.*/){

@itag = m/[&?](itag\=[0-9]*)/;

@range = m/[&?](range\=[^\&\s]*)/;

@cpn = m/[&?]cpn\=([^\&\s]*)/;

@mime = m/[&?](mime\=[^\&\s]*)/;

@id = m/[&?]id\=([^\&\s]*)/;

if (defined(@cpn[0])){

if (-e "/tmp/@cpn"){

open FILE, "/tmp/@cpn";

@id = <FILE>;

close FILE;}

$out="OK store-id=http://video-srv.squid.internal/id=@id@mime@range";

} else {

$out="ERR";

print $logfh "$timenow"."out: $a $out\n" if ($debug>=1);

print "$a $out\n";

close $logfh if ($debug);

Tested untuk cache Facebook HTTPS dan youtube. Silakan di copas kalo pengen :D

Facebook

Twitter

Google+

You may also like:

Instalasi Modul Perl Readbackwards di IPFire

SSL Bump dan Cache HTTPS Dengan Webproxy IPFire

Catatan Pribadi : Flashback IPFire 2012 2014

Mengenal Lebih Jauh Tentang Web Content Caching

Konfigurasi Minimum Squid Proxy

Incoming search:

4 Comments

zagi October 1, 2014 at 2:16 am

Bang fazar gmn caranya set $debug=1; pada storeid, dan buat file /tmp/storeid.log yang sudah di
chown ke user proxy/squid nya?mohon pencerahanya..thx sangat membantu
REPLY

fazar October 1, 2014 at 7:58 am

pertama set $debug=1; (bisa menggunakan editor seperti vi, nano, dsb) selanjutnya ikuti step berikut
di ssh :
# touch /tmp/storeid.log
# chown squid:squid /tmp/storeid.log
# squid -k reconfigure
selesai, silakan tail /tmp/storeid.log :)