This should be simple, but it's got me stumped.
First, plug the following URL into any browser (Mozilla, IE, Thunderbird, Safari, or whatever, all one line in the URL panel):
http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest
<Address ID="0"><Address1></Address1><Address2>6406 Ivy
Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>
You should get back the following response from the shippingapis.com machine:
<Error> <Number>80040b19</Number> <Description>XML Syntax Error: Error getting USERID attribute.</Description> <Source>UspsCom::DoAuth</Source> </Error>
And, this is how it should be. And, I would like to duplicate this behavior via Squeak code. So, I try, for example, the following:
HTTPClient httpGetDocument: 'http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest><Address ID="0"><Address1></Address1><Address2>6406 Ivy Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>'
I would think that the above would be the programmatic equivalent of the URL I typed into Thunderbird (I also tried it with Safari, and both Safari and Thunderbird gave me the same results). But when I do it programmatically in Squeak, I get back this response:
HTTP/1.1 400 Bad Request Server: Microsoft-IIS/5.0 Date: Tue, 27 Dec 2005 03:46:44 GMT Content-Type: text/html Content-Length: 87<html><head><title>Error</title></head><body>The parameter is incorrect. </body></html>
I was expecting the "XML Syntax Error" response that Thunderbird (as well as Safari) gave me instead of this nonsense. But I instead got back something totally different. Why?
OK, let's try Scamper. So, I plug in the URL I mentioned, and I get the following response in Scamper:
error occured retrieving http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest
<Address ID="0"><Address1></Address1><Address2>6406 Ivy
Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>: HTTP/1.1 400 Bad Request Server: Microsoft-IIS/5.0 Date: Tue, 27 Dec 2005 03:50:47 GMT Content-Type: text/html Content-Length: 87<html><head><title>Error</title></head><body>The parameter is incorrect. </body></html>
So, Scamper is also giving me something different from the "XML Syntax Error" response I was expecting. Why?
I've tried every combination of GET and/or POST I could think of. I just don't understand why Squeak is giving me a different result than Thunderbird gives me. Anybody know?
In case you are curious, if I include my login ID as an XML arg of the URL, the shippingapis.com machine will validate my address(es) that I pass to it. I just didn't think it would be a good idea to post my login ID. Besides, it is unnecessary to do so, because if I solve the above problem, I will solve my *real* problem, and will be able to programmatically validate any US-based address.
Anybody know why Squeak is giving me different results than the web browsers do?
Nevin
Nevin -
Have you watched the HTTP request/response interaction using Ethereal? I find that tool to be a very helpful complement to digging into networking code.
Brent Vukmer wrote:
Nevin -
Have you watched the HTTP request/response interaction using Ethereal? I find that tool to be a very helpful complement to digging into networking code.
I'll try it. Unfortunately, I'm Mac based. So, I'll have to dig up another machine somewhere. But I'll give it a go.
In the meantime, if anybody else has any other ideas, it would be appreciated.
Nevin
Nevin Pratt wrote:
Brent Vukmer wrote:
Have you watched the HTTP request/response interaction using Ethereal? I find that tool to be a very helpful complement to digging into networking code.
I'll try it. Unfortunately, I'm Mac based. So, I'll have to dig up another machine somewhere. But I'll give it a go.
Nevin, try using tcpdump from a Terminal. It comes w/ OS X.
You'll have to use sudo to run it as root ("sudo tcpdump").
Bob
Bob Courchaine wrote:
Nevin Pratt wrote:
Brent Vukmer wrote:
Have you watched the HTTP request/response interaction using Ethereal? I find that tool to be a very helpful complement to digging into networking code.
I'll try it. Unfortunately, I'm Mac based. So, I'll have to dig up another machine somewhere. But I'll give it a go.
Nevin, try using tcpdump from a Terminal. It comes w/ OS X.
You'll have to use sudo to run it as root ("sudo tcpdump").
Bob
I tried that. It wasn't particularly helpful. For example, while running tcpdump, when I type the following URL into my (Thunderbird) browser:
http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest
<Address ID="0"><Address1></Address1><Address2>6406 Ivy
Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>
tcpdump gives me the following:
10:56:05.224754 IP 192.168.0.25.56308 > utscfwhm01.ut.sprintbbd.net.domain: 26942+ A? testing.shippingapis.com. (42) 10:56:05.345274 IP utscfwhm01.ut.sprintbbd.net.domain > 192.168.0.25.56308: 26942* 1/3/3 A[|domain] 10:56:05.346627 IP 192.168.0.25.50582 > testing.shippingapis.com.http: S 1878312729:1878312729(0) win 65535 <mss 1460,nop,wscale 0,nop,nop,timestamp 1507361664 0> 10:56:05.414864 IP testing.shippingapis.com.http > 192.168.0.25.50582: S 2571541148:2571541148(0) ack 1878312730 win 17520 <mss 1460,nop,wscale 0,nop,nop,timestamp 0 0> 10:56:05.414968 IP 192.168.0.25.50582 > testing.shippingapis.com.http: . ack 1 win 65535 <nop,nop,timestamp 1507361664 0> 10:56:05.415152 IP 192.168.0.25.50582 > testing.shippingapis.com.http: P 1:723(722) ack 1 win 65535 <nop,nop,timestamp 1507361664 0> 10:56:05.528382 IP testing.shippingapis.com.http > 192.168.0.25.50582: P 1:101(100) ack 723 win 16798 <nop,nop,timestamp 112599815 1507361664> 10:56:05.529178 IP testing.shippingapis.com.http > 192.168.0.25.50582: FP 101:258(157) ack 723 win 16798 <nop,nop,timestamp 112599815 1507361664> 10:56:05.529229 IP 192.168.0.25.50582 > testing.shippingapis.com.http: . ack 259 win 65535 <nop,nop,timestamp 1507361664 112599815> 10:56:05.529436 IP 192.168.0.25.50582 > testing.shippingapis.com.http: F 723:723(0) ack 259 win 65535 <nop,nop,timestamp 1507361664 112599815> 10:56:05.599318 IP testing.shippingapis.com.http > 192.168.0.25.50582: . ack 724 win 16798 <nop,nop,timestamp 112599816 1507361664>
Then, in Squeak, when I enter that same URL as an argument to "HTTPClient httpGetDocument:", tcpdump gives me the following:
11:00:42.637048 IP 192.168.0.25.56324 > utscfwhm01.ut.sprintbbd.net.domain: 4024+ PTR? 12.0.0.224.in-addr.arpa. (41) 11:00:42.693034 IP utscfwhm01.ut.sprintbbd.net.domain > 192.168.0.25.56324: 4024* 1/3/9 PTR[|domain] 11:00:43.055624 IP 192.168.0.25.49271 > testing.shippingapis.com.http: S 849893481:849893481(0) win 65535 <mss 1460,nop,wscale 0,nop,nop,timestamp 1507362219 0> 11:00:43.193739 IP testing.shippingapis.com.http > 192.168.0.25.49271: S 1243403613:1243403613(0) ack 849893482 win 17520 <mss 1460,nop,wscale 0,nop,nop,timestamp 0 0> 11:00:43.193830 IP 192.168.0.25.49271 > testing.shippingapis.com.http: . ack 1 win 65535 <nop,nop,timestamp 1507362220 0> 11:00:43.555171 IP 192.168.0.25.49271 > testing.shippingapis.com.http: P 1:366(365) ack 1 win 65535 <nop,nop,timestamp 1507362220 0> 11:00:43.636319 IP testing.shippingapis.com.http > 192.168.0.25.49271: F 225:225(0) ack 366 win 17155 <nop,nop,timestamp 112585161 1507362220> 11:00:43.636409 IP 192.168.0.25.49271 > testing.shippingapis.com.http: . ack 1 win 65535 <nop,nop,timestamp 1507362220 0> 11:00:43.637288 IP testing.shippingapis.com.http > 192.168.0.25.49271: P 1:225(224) ack 366 win 17155 <nop,nop,timestamp 112585161 1507362220> 11:00:43.637322 IP 192.168.0.25.49271 > testing.shippingapis.com.http: . ack 226 win 65535 <nop,nop,timestamp 1507362220 112585161> 11:00:44.376669 IP 192.168.0.25.49271 > testing.shippingapis.com.http: R 366:366(0) ack 226 win 65535
As far as I can tell, that doesn't tell me anything about why the Microsoft IIS server at "testing.shippingapis.com" is treating the two requests differently-- why it works fine when my browser is the client, but returns a "400 Bad Request" error when Squeak is the client.
Nevin
Nevin Pratt wrote:
As far as I can tell, that doesn't tell me anything about why the Microsoft IIS server at "testing.shippingapis.com" is treating the two requests differently-- why it works fine when my browser is the client, but returns a "400 Bad Request" error when Squeak is the client.
You will need to look at what the browsers are sending. One way would be to open a listening socket in squeak on port 80 and then connect to it with your browser with a url like http://localhost/ShippingAPITest.dll?API=Verify&XML=<AddressVa ...etc
Or even before that look at how squeak actually parses the URL.
Michael
I think if you look at the man page for TCPDUMP you'll find it can dump the contents of the tcp packets, versus the headers, then you can see exactly what the browser is sending and getting.
On 27-Dec-05, at 10:30 AM, Michael Rueger wrote:
Nevin Pratt wrote:
As far as I can tell, that doesn't tell me anything about why the Microsoft IIS server at "testing.shippingapis.com" is treating the two requests differently-- why it works fine when my browser is the client, but returns a "400 Bad Request" error when Squeak is the client.
You will need to look at what the browsers are sending. One way would be to open a listening socket in squeak on port 80 and then connect to it with your browser with a url like http://localhost/ShippingAPITest.dll?API=Verify&XML=<AddressVa ...etc
Or even before that look at how squeak actually parses the URL.
Michael
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
Or you can use tcpflow: http://www.entropy.ch/software/macosx/#tcpflow and you can see the content of tcp packet. I am using the 0.21 package and it work well on 10.3 and 10.4 Don't forget the -c flag to see content on the console.
Alain
Le 27 déc. 05, à 21:57, John M McIntosh a écrit :
I think if you look at the man page for TCPDUMP you'll find it can dump the contents of the tcp packets, versus the headers, then you can see exactly what the browser is sending and getting.
On 27-Dec-05, at 10:30 AM, Michael Rueger wrote:
Nevin Pratt wrote:
As far as I can tell, that doesn't tell me anything about why the Microsoft IIS server at "testing.shippingapis.com" is treating the two requests differently-- why it works fine when my browser is the client, but returns a "400 Bad Request" error when Squeak is the client.
You will need to look at what the browsers are sending. One way would be to open a listening socket in squeak on port 80 and then connect to it with your browser with a url like http://localhost/ShippingAPITest.dll?API=Verify&XML=<AddressVa ...etc
Or even before that look at how squeak actually parses the URL.
Michael
--
==== John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================= ====
Nevin Pratt wrote:
http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest
<Address ID="0"><Address1></Address1><Address2>6406 Ivy
Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>
Yesterday, I tried this URL in Firefox running on XP Home. It kept coming back with a dialog to save or open "ShippingAPITest.dll". I chose not to save it, and assumed that maybe someone else would have an answer for you. Anyhow, is it possible that you have this .dll already on your system, but Squeak has no way to make use of it?
Yanni Chiu wrote:
Nevin Pratt wrote:
http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest
<Address ID="0"><Address1></Address1><Address2>6406 Ivy
Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>
Yesterday, I tried this URL in Firefox running on XP Home. It kept coming back with a dialog to save or open "ShippingAPITest.dll". I chose not to save it, and assumed that maybe someone else would have an answer for you. Anyhow, is it possible that you have this .dll already on your system, but Squeak has no way to make use of it?
No.
Save it. It is a text file. Look at the text. That is the response text, in the file that you are saving.
I don't understand web protocols well enough to know why the response is coming back via a file named "ShippingAPIText.dll". But, the file actually contains the text response that I am looking for.
How do I get Squeak to do this?
Thanks, Yanni!
Nevin
I didn't dig very far but it seems Firefox's issue is that doesn't know what to do w/ a ".dll" file.
I downloaded the response, changed the extension to .xml and Firefox opened it formatted as XML, complaining, "This XML file does not appear to have any style information associated with it. The document tree is shown below."
Just for fun, I added a prolog declaring the document as XML (<?xml version = "1.0"?>). Then I changed the extension back to .dll and tried to Open the File in Firefox. It still didn't know how to handle it, asking to download it again.
So in this case the issue is with the client.
From working w/ web services, I'd not expect to display this in a web
browser. I'd save it to the filesystem and parse, etc it from there.
Just because IE displays it w/o complaining doesn't mean it's typical and expected practice.
I deleted your original question, Nevin, so I'm not sure what you're trying to do. But I'd suggest it's not the GET mechanism that's the problem, it's the handling of the response.
Bob
Nevin Pratt wrote:
Yanni Chiu wrote:
Nevin Pratt wrote:
http://testing.shippingapis.com/ShippingAPITest.dll?API=Verify&XML=<AddressValidateRequest
<Address ID="0"><Address1></Address1><Address2>6406 Ivy
Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>
Yesterday, I tried this URL in Firefox running on XP Home. It kept coming back with a dialog to save or open "ShippingAPITest.dll". I chose not to save it, and assumed that maybe someone else would have an answer for you. Anyhow, is it possible that you have this .dll already on your system, but Squeak has no way to make use of it?
No.
Save it. It is a text file. Look at the text. That is the response text, in the file that you are saving.
I don't understand web protocols well enough to know why the response is coming back via a file named "ShippingAPIText.dll". But, the file actually contains the text response that I am looking for.
How do I get Squeak to do this?
Thanks, Yanni!
Nevin
Bob Courchaine wrote:
I didn't dig very far but it seems Firefox's issue is that doesn't know what to do w/ a ".dll" file.
I downloaded the response, changed the extension to .xml and Firefox opened it formatted as XML, complaining, "This XML file does not appear to have any style information associated with it. The document tree is shown below."
Just for fun, I added a prolog declaring the document as XML (<?xml version = "1.0"?>). Then I changed the extension back to .dll and tried to Open the File in Firefox. It still didn't know how to handle it, asking to download it again.
Actually, I don't care to open the file in Firefox, or any other browser, and I don't care about files.
I just want to hit that URL in Squeak, and get the response into Squeak.
If you put the URL in Firefox, it is correct that Firefox thinks it is downloading a file. But the file that it downloads is a text file that contains the response.
I want to retrieve that response, via the URL, totally inside of Squeak, without dealing with files. It's that simple.
Or so I would think.
Nevin
I haven't been using HttpClient, but I've been fetching documents using something like
'http://www.foo.com' asUrl retrieveContents
which gets you an instance of MimeDocument. I don't know if that helps you.
On Dec 28, 2005, at 10:02 AM, Nevin Pratt wrote:
Bob Courchaine wrote:
I didn't dig very far but it seems Firefox's issue is that doesn't know what to do w/ a ".dll" file.
I downloaded the response, changed the extension to .xml and Firefox opened it formatted as XML, complaining, "This XML file does not appear to have any style information associated with it. The document tree is shown below."
Just for fun, I added a prolog declaring the document as XML (<?xml version = "1.0"?>). Then I changed the extension back to .dll and tried to Open the File in Firefox. It still didn't know how to handle it, asking to download it again.
Actually, I don't care to open the file in Firefox, or any other browser, and I don't care about files. I just want to hit that URL in Squeak, and get the response into Squeak.
If you put the URL in Firefox, it is correct that Firefox thinks it is downloading a file. But the file that it downloads is a text file that contains the response.
I want to retrieve that response, via the URL, totally inside of Squeak, without dealing with files. It's that simple.
Or so I would think.
Nevin
Nevin Pratt wrote:
I want to retrieve that response, via the URL, totally inside of Squeak, without dealing with files. It's that simple.
?
I thought your issue was w/ a client you were trying to use, like Celeste.
In a Workspace, Inspect "<your long URL> asUrl retrieveContents".
A MIMEDocument is returned. Look at the "Contents" and you'll see your error message.
Bob Courchaine wrote:
Nevin Pratt wrote:
I want to retrieve that response, via the URL, totally inside of Squeak, without dealing with files. It's that simple.
?
I thought your issue was w/ a client you were trying to use, like Celeste.
In a Workspace, Inspect "<your long URL> asUrl retrieveContents".
A MIMEDocument is returned. Look at the "Contents" and you'll see your error message.
But that's the original, wrong, error message. The problem is that your request is not properly URL-encoded. The easiest way to achieve this (which is also much easier to use programmatically given an XML request string) looks like this:
HTTPSocket httpGetDocument: 'http://testing.shippingapis.com/ShippingAPITest.dll' args: ({'API' -> #('Verify'). 'XML' -> #('<AddressValidateRequest ><Address ID="0"><Address1></Address1><Address2>6406 Ivy Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>') } as: Dictionary) accept: 'text/xml'
Note that the dictionary values must be arrays of strings, not simple strings. This is a little awkward, but it's how the class is implemented. I've tested it, it works.
Cheers, Hans-Martin
But that's the original, wrong, error message. The problem is that your request is not properly URL-encoded. The easiest way to achieve this (which is also much easier to use programmatically given an XML request string) looks like this:
HTTPSocket httpGetDocument: 'http://testing.shippingapis.com/ShippingAPITest.dll' args: ({'API' -> #('Verify'). 'XML' -> #('<AddressValidateRequest ><Address ID="0"><Address1></Address1><Address2>6406 Ivy Lane</Address2><City>Greenbelt</City><State>MD</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>')
} as: Dictionary) accept: 'text/xml'
Note that the dictionary values must be arrays of strings, not simple strings. This is a little awkward, but it's how the class is implemented. I've tested it, it works.
Cheers, Hans-Martin
Your code works. Thanks, Hans!
Also, your comment about using "asUrl retrieveContents" producing the wrong results is spot on. Your code is what was needed. Your code lets me use the USPS Web Tools API, to do address validation and more.
If anybody else wants to know more about doing this, I've pasted an email from USPS below, and it gives further details. In the email, I've replaced my true Web Tools User ID with x's, though, for obvious reasons. You would obviously have to contact USPS yourself to get your own user ID.
Thanks, everyone!
Nevin
Thank you for registering for the U. S. Postal Service's Web Tools Application Program Interfaces (APIs). We are providing you with a User ID that serves multiple purposes, as explained below.
Your Web Tools User ID is: xxxxxxxxxxxx
The latest versions of the technical documentation, including the Administrative Guide, are available from USPS.com at: www.usps.com/webtools. They provide documentation and sample code for implementing our APIs. This documentation is in PDF and HTML format. In order to open and view this document in PDF you must have Adobe Acrobat Reader installed on your system. You may download this software, at no cost, from http://www.adobe.com/products/acrobat/readstep2.html.
IMPORTANT NOTE: The current documentation refers to a password. Please ignore all references to password in the documentation. Password is no longer used or required.
Your Web Tools User ID to integrate USPS Web Tools is provided above. The User ID is used for testing your implementation of the API(s). With this ID, you may begin sending calls to the test server. The address to the test server is: testing.shippingapis.com and the path is /ShippingAPITest.dll. Use this information in combination with your User ID and your XML string to send a request to the USPS servers. For more details, refer to the programming guides (located at www.usps.com/webtools) for the specific API you are integrating.
A sample test request would look like: "http://testing.shippingapis.com/ShippingAPITest.dll?API=%5BAPI_Name%5D&X... [XML_String_containing_User_ID]"
When you have completed your testing, email the USPS Internet Customer Care Center (ICCC). They will switch your profile to allow you access to the production server and will provide you with the production URL.
The ICCC is staffed from 7:00AM to 11:00PM Eastern Time.
E-mail: icustomercare@usps.com Telephone: 1-800-344-7779
IMPORTANT NOTICE REGARDING USER IDThe Web Tools User ID provided is for you and your company to use when requesting data via the Internet from the U.S. Postal Service API servers. This unique User ID cannot be shared with others outside your organization, nor is it to be packaged and distributed or sold to other individuals, businesses or e-commerce web site entities. As per the Terms and Conditions of Use Agreement you agreed to during the Web Tools registration process, you are responsible to maintain the confidentiality of your User ID as specified. You may not package any APIs with your User ID for resale or distribution to others. The U.S. Postal Service does not prohibit the reuse and/or distribution of the API documentation (User's Guide) with sample code in order to generate awareness, encourage use or provide ease-of-use to customers or affiliates.
Warning - If the U.S. Postal Service discovers use of the same User ID from more than one web site, all users will be subject to loss of access to the USPS production server and/or termination of the licenses granted under the Terms and Conditions of Use. For more information regarding the USPS Web Tools User ID policy, or for questions regarding distribution of documentation, send email to icustomercare@usps.com.
Thank you for helping the U.S. Postal Service provide new services to our shipping customers.
Sincerely,
The Internet Shipping Solutions Team
USPS Internet Customer Care Center icustomercare@usps.gov 7:00AM - 11:00PM EST
squeak-dev@lists.squeakfoundation.org
-
Alain Fischer -
Bob Courchaine -
Brent Vukmer -
Hans-Martin Mosner -
John M McIntosh -
Michael Rueger -
Nevin Pratt -
Todd Blanchard -
Yanni Chiu