Note: Python: Follow 301 redirect

      No Comments on Note: Python: Follow 301 redirect
import urllib.parse as urlparse
import http.client as httplib
#----------------------------------------------------------------------
def resolve_http_redirect(url, depth=0):
    """http://www.zacwitte.com/resolving-http-redirects-in-python
    """
    if depth > 10:
        raise Exception("Redirected "+depth+" times, giving up.")
    o = urlparse.urlparse(url,allow_fragments=True)
    conn = httplib.HTTPConnection(o.netloc)
    path = o.path
    if o.query:
        path +='?'+o.query
    conn.request("HEAD", path)
    res = conn.getresponse()
    headers = dict(res.getheaders())
    if 'Location' in headers and headers['Location'] != url:
        return resolve_http_redirect(headers['Location'], depth+1)
    else:
        return url

 

笔记:找视频网站解析的思路

      No Comments on 笔记:找视频网站解析的思路

最近给you-get写了不少解析,但是自己的repo为了更新经常删,所以留不下记录,只能在主repo里面留名了。https://github.com/soimort/you-get/graphs/contributors
例子仅供参考,肯定不完整,主要抛砖引玉,想到哪写到哪。

  •  直接网页截取?(FC2Video)
    • 用exec直接变dict?(pixnet)
      • encode是否全替换?(pixnet)
  • Flash引出的API?(很多,例如,Bilibili)
    • API参数是否可调?(pixnet,fun.tv)
      • 参数可否不加?(pixnet,Bilibili)
      • key是否可以申请?(VImeo,Bilibili)
    • 是否涉及hash?(绝大多数)
      • 反编译源文件?(iQiyi and shitloads of them ,letvcloud)
  • HTML5?(Weibo Miaopai)
    • API?移动端?(Fun.tv)
    • API是否相同?(Fun.tv,Letvcloud)
    • 是否某些加密不可进行?(Letvcloud)
  • 移动网站?(Pandora)
    • 是否直接页内?(Pandora)
    • iOS?Android?
    • 清晰度是否相同?
  • 移动端?
    • 是否有API?(Vimeo,Bilibili)
      • 是否HTTPS?(Vimeo)
        • 可否MITM/SSLStrip?(Vimeo)
    • 是否需要hash?(很多)
      • Android反编译?(另一个东西,Chrome DCP Srandalone https://github.com/cnbeining/Chrome-Data-Compression-Proxy-Standalone-Python 用了)
  • 反加密?
    • 特殊工具?(Letvcloud,Bilibili)
    • 其他工具/网站的提示?
    • 特殊referer/UA?特殊XFF/X-Real-IP?(Bilibili)
    • 反侦察?(iQiyi)

重装系统

      No Comments on 重装系统

mariadb把数据库弄死了,重装。
这东西弄得整个服务器全死了:重装的机器都死了。
把数据库文件扯出来,强行挂载在别的机器上,强行导出数据库,这才恢复了数据。
临时上个SSL证书,未来上letsencrypt。

立此存照

      1 Comment on 立此存照


Two days ago the police came to me and wanted me to stop working on this. Today they asked me to delete all the code from GitHub. I have no choice but to obey.
I hope one day I'll live in a country where I have freedom to write any code I like without fearing.
I believe you guys will make great stuff with Network Extensions.
Cheers!
 
https://web.archive.org/web/20150822042959/https://github.com/shadowsocks/shadowsocks-iOS/issues/124
 
Lest we forget.

应求写一个刷逼站直播在线人数的脚本/A simple script to get lots of viewers of live.bilibili.com

应求,就这样。
什么Queue啊都不用了,投入list的怀抱吧。
https://gist.github.com/cnbeining/6b2273d7e332f29193d0

#!/usr/bin/env python
#coding:utf-8
# Author:  Beining --<cnbeining#gmail.com>
# Purpose: A simple script to get lots of viewers of Bilibili Live
# Created: 08/11/2015
# Error report: http://www.cnbeining.com/?p=952
# https://github.com/cnbeining  somewhere within my gists
import sys
import time
import getopt
from multiprocessing import Process
from websocket import create_connection, WebSocketConnectionClosedException
#----------------------------------------------------------------------
def fake_connector(cid):
    """"""
    try:
        ws = create_connection('ws://livecmt.bilibili.com:88/{cid}'.format(cid = cid))
        while 1:
            time.sleep(5)
            a = ws.recv()
    except WebSocketConnectionClosedException:
        return -1
    except Exception as e:
        print(e)
        return 0
    finally:
        return
#----------------------------------------------------------------------
def main(cid, thread_number):
    """"""
    #Get a list of threads
    process_list = [Process(target=fake_connector, args=((cid, ))) for i in xrange(int(thread_number))]  #Queue? Your ass
    [i.start() for i in process_list]  #ignite every one
    try:
        while 1:
            alive_list = [i.is_alive() for i in process_list]
            print('Active thread: ' + str(len(alive_list)))
            death_position_list = [i for i, x in enumerate(alive_list) if x == False]
            if len(death_position_list) > 0:  #someone died
                print('Some died, adding {COUNT} new thread'.format(COUNT = len(death_position_list)))
                for i in death_position_list:
                    del process_list[i]
                    process_list.append(Process(target=fake_connector, args=((cid, ))))
                    process_list[-1].start()
            time.sleep(3)
    except Exception as e:
        print(e)
        for i in process_list:
            try:
                i.terminate()
            except:
                pass
        exit()
#----------------------------------------------------------------------
def usage():
    """"""
    print('''Use as:
    -c: cid, room number
    -t: thread number
    Press Ctrl+C to exit.
    ''')
if __name__=='__main__':
    argv_list = sys.argv[1:]
    try:
        opts, args = getopt.getopt(argv_list, "hc:t:",
                                   ['help', "cid=", 'thread_number='])
    except getopt.GetoptError:
        usage()
        exit()
    for o, a in opts:
        if o in ('-h', '--help'):
            usage()
            exit()
        if o in ('-c', '--cid'):
            cid = a
        if o in ('-t', '--thread_number'):
            thread_number = int(a)
    print('Getting room {cid} {thread_number} viewers...'.format(cid = cid, thread_number = thread_number))
    main(cid, thread_number)

 

关于B站无法访问/没图/页面错乱的问题的改host大法

这点破事八百万人一天问我八百万遍。
解决方案:
IE无人权 Opera类似Chrome 国产垃圾浏览器赶快都丢出去;
英文版,其他照猫画虎。
1. 审查元素(开发者工具)
Chrome: FF: Safari:
 
选择网络,Network:
Chrome:FF: Safari:
 
2. 刷新页面,看你是怎么死的:
Chrome: 其他两个类似。
 
3. 看看谁红了?红了的,域名是多少?
例如,上面Chrome这张红图,涉及的域名包括:
static.hdslb.com
i1.hdslb.com
i2.hdslb.com
你也有可能遇到别的。
4. 全国ping之:
我个人喜欢 http://www.17ce.com/
类似的网站使用类似。


 
5. 你已经有一堆IP地址了。
挑几个,进入命令提示符,试着ping一下:
无论平台,命令为:
ping (IP地址)

OSX的长这样,其他平台的类似。
6. 修改hosts:
Windows: https://support.microsoft.com/en-us/kb/923947
OSX:注意有可能需要root
其他平台:你都不应该需要看我这个东西
7. 保存,重启浏览器,试试是不是好了:
7.1 没好:清除DNS缓存试试
7.1.1 还没好:换个IP地址试试
 
以上。