Announcement

Collapse
No announcement yet.

Downloading a web page with telnet

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Downloading a web page with telnet

    Hey all,
    Anyone here remember the commands needed to download a web page with telnet?

    TAI.
    --bc,

  • #2
    Originally posted by big chopper
    Hey all,
    Anyone here remember the commands needed to download a web page with telnet?
    Did you try:

    Really basic selection without special extra options can work:
    Code:
    $ telnet 127.0.0.1 80
    Trying 127.0.0.1...
    Connected to 127.0.0.1.
    Escape character is '^]'.
    GET /PATH/TO/TXT/OR/HTML/FILE.txt.or.html HTTP/1.0^M^M
    Though, you should be able to use ^J^J instead of ^M^M or press the "RETURN" key twice.

    But why telnet? Why not use "wget" or "sitecopy" ? They have many more features for conversion of links to absolut/dynamic name of referring site (for check of internal referral) and web client browser version.

    wget and sitecopy will also work on servers that have multiple domains sharing a single IP address.

    [Added content:]
    More advanced features when connecting:
    Code:
    $ telnet 127.0.0.1 80
    Trying 127.0.0.1...
    Connected to 127.0.0.1.
    Escape character is '^]'.
    GET /PATH/TO/TXT/OR/HTML/FILE.txt.or.html HTTP/1.0
    Host: NAMEOFHOST.YOU.WANT.TO.CONNECT.IF.MULTI.HOMED.TLD
    Accept: text/html, text/plain, text/*
    Accept: ANY_OTHER_MIME_TYPES_COMMA_SEPARATED
    Accept: MULTIPLE_ACCEPT_LINES_OK
    Accept-Language: en
    User-Agent: I-AM-CORNHOLIO/1.0 IP-FOR-MY-TCP NCSA-Mosaic^M^M
    You can end each line with a ^M or ^J, but you signify you are done entering options with a ^M^M or ^J^J or try pressing the return key twice.
    Other options exist. Related docs:
    http://www.w3.org/Protocols/rfc1945/rfc1945
    http://www.w3.org/Protocols/rfc2616/rfc2616.html
    Last edited by TheCotMan; April 20, 2005, 21:58.

    Comment


    • #3
      For what ever reason, that doesn't work. I need to specify a full URI
      telnet http://www.domain.com 80
      GET http://www.domain.com/index.php
      --bc,

      Comment


      • #4
        Originally posted by big chopper
        For what ever reason, that doesn't work. I need to specify a full URI
        telnet http://www.domain.com 80
        GET http://www.domain.com/index.php
        --bc,
        It is likely they rely upon the client to pass the name of the server for which they want to see content. Apache's option for virtualhost uses this method to allow a single IP address to run a single web server that serves many different domains. However,this means any client not specifying the "Host:" option will likely fail to get its content.

        Your telnet client appears to have modified your "http://www.domain.com" to "www.domain.com" for you. It is good that the server respected the GET as you formatted it without a specification of what version HTTP you were using and the conventional "Host:" option, but instead relying upon the host as handed int he GET..:-)

        (I added the content on some of the extra options to the first reply as well as the RFC that relate to this. This is stated so people know you replied before the [added content] in my first reply was included and it was my fault for not editing faster :-)
        Last edited by TheCotMan; April 20, 2005, 22:10.

        Comment


        • #5
          Originally posted by big chopper
          For what ever reason, that doesn't work. I need to specify a full URI
          telnet http://www.domain.com 80
          GET http://www.domain.com/index.php
          --bc,
          Is this going to be part of a script? Why not use curl/lwp/fetch/wget?
          "Never Underestimate the Power of Stupid People in Large Groups"

          Comment


          • #6
            Originally posted by hackajar
            Is this going to be part of a script? Why not use curl/lwp/fetch/wget?
            OK, my actual goal is to use telnet to get mail from a pop3 mail server. I wanted to download a web page first, for a basic understanding of telnet.

            That (telnet) exercise was groundwork for a PHP/CURL script that programatically reads/deletes email from a POP3 mail server. I had to learn the POP3 commands first, telnet was a good place to experiment.

            Thanks everyone for your help.
            --bc,
            Last edited by big chopper; April 23, 2005, 17:02.

            Comment


            • #7
              Pop3

              1.) Just ask, this question could have been aswered days ago

              2.) These are the droids you are looking for

              3.)
              %telent 127.0.0.1 110

              >user myuser
              USER OK
              >pass mypass
              LOGGED IN
              >list
              1 512
              2 86
              3 9806
              > top 1 512
              Header....bla bla bla
              <Subject>your V|@gra request
              Thankyou for buying our great product!!!


              KEY:
              % = shell command line (remove % and type that command)
              > = POP3 Command line (remove > and type that command)
              Anything else is returned results
              "Never Underestimate the Power of Stupid People in Large Groups"

              Comment


              • #8
                php pop3

                by hand....

                <?php

                $Array = array("user"=>"user myuser", "pass"=>"pass pass");

                //We love d socket!
                $socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
                //We open d socket!
                socket_connect($socket, "127.0.0.1", "110");
                //Send username
                socket_write($socket, $Array['user'], strlen($Array['user']));
                //Check result came back
                while ($out = socket_read($socket, 2048)) {
                $result .= $out;
                }
                //Send Pass
                if($result)socket_write($socket, $Array['pass'], strlen($Array['pass']));
                $result = false;
                while ($out = socket_read($socket, 2048)) {
                $result .= $out;
                }
                //Check Auth OK
                if(!preg_match("/OK/", $result)) die("Authentication faild!");

                //Start Getting Mail

                bla bla bla

                ?>
                "Never Underestimate the Power of Stupid People in Large Groups"

                Comment


                • #9
                  Originally posted by hackajar
                  by hand....
                  <?php
                  <snip> ... </snip>
                  ?>
                  Yep, that's pretty much what I did.

                  When you execute a POP3 LIST command, it returns a message id and a size.
                  LIST
                  +OK
                  1 2400
                  2 5432
                  (etc...)

                  Initially I used the returned size value to let me know when the entire message was read. In actual practice, however, I found the size value returned by LIST has little resemblance to the actual size of the message. (Is the value in octal or something?) A better technique is to use the POP3 end of multi-line message indicator (a ".\n")--which is what I should have done from the beginning, I suppose.

                  --bc,

                  Comment

                  Working...
                  X