Developer Blog http://developers.eveonline.com/ 2024-03-19T01:59:16Z http://developers.eveonline.com/blog/article/the-return-of-the-developers-blog/ The return of the Developers blog! 2024-02-27T00:00:00Z 2024-02-27T00:00:00Z CCP Zelus 2119788568 Greetings, developers!

Today's post covers Plans for developer-related news, an SDE update and clearing up an old "Temporary" alert that existed here.

News update:

We know it's been a long time since this page was updated, but it's time we have a home for third-party developer-related content once again.

We had been using a combination of the forums, Tweetfleet Slack and Discord to convey messages, but having an RSS news feed enabled post and a clean-from-clutter central location for information was the most requested thing from the community, so resurrecting this blog, as it has all the desired functionality seems to be the best avenue to facilitate that request!

SDE:

This is a two-fold announcement - the SDE with the updates from the balance patch from 20 February 2024 is now available at https://developers.eveonline.com/resource

The SDE has also evolved. I had a note in my calendar to check if a new SDE deployment was required in the middle of each month since taking over the project and to decide to either postpone another month or create a deployment. There are currently plans to develop this into a semi-automated process now instead, where a new SDE can be created at the end of each month automatically whilst still retaining the ability to manually deploy an update in between those times, so we can ensure that in the time following an expansion that we don't have to wait until the end of the month to get the SDE, but also if there are any minor changes, that the end of the month update should facilitate this. Again, once this is actioned, and in place, we'll relay that information here!

Temporary Blackout of ESI Bookmarks endpoint

In this article, we mentioned in 2019 that there would be a "Temporary" blackout of the bookmark endpoints when we migrated from the old bookmark system to shareable/ACL-based bookmarks. Since then, the bookmarks endpoint has been returning empty lists if used. There are currently no plans to restore this endpoint due to how the ACL system is implemented; the ESI system is not set up in such a way as to support this type of request. If this changes in the future, we will convey that information here.

See you in the next update!

o7

]]>
The return of the Developers blog! the-return-of-the-developers-blog
http://developers.eveonline.com/blog/article/sso-endpoint-deprecations-2/ SSO Endpoint Deprecations 2021-10-07T13:06:54Z 2021-10-07T13:06:54Z CCP Convict 2115597014 Hi developer friends!

We'd like to inform you of upcoming SSO changes which might affect your application if you are developing 3rd party apps using ESI or using the EVE SSO for user authentication.

We'll be making the following changes on November 1st 2021.

Deprecating obsolete endpoints

The old OAuth endpoints oauth/authorize, oauth/token and oauth/verify will be deprecated. You should move your application over to v2/oauth/authorize and v2/oauth/token instead. The oauth/verify endpoint will also be deprecated, since the v2 endpoints return a JWT token, enabling your applications to validate the JWT tokens without having to make a request to the SSO for each token. Applications can fetch the required EVE SSO metadata from https://login.eveonline.com/.well-known/oauth-authorization-server to be able to validate JWT token signatures client-side.

The deprecated endpoints might be removed at any given time after November 1st 2021!

What about our old refresh tokens?

Existing applications can do a one-way-upgrade on their existing refresh tokens by sending them to the v2/oauth/token instead of oauth/token endpoint to upgrade them to v2 tokens. Make sure to replace the stored refresh token with the one returned in the response! Once an old refresh token is upgraded to a v2 token, the old token is invalidated and no longer usable. This upgrade mechanism will only be active for two weeks after the release.

You can tell the new refresh tokens apart from the old ones as they are a 24 character strings (16 bytes Base64 encoded).

V2 Refresh token example: MDEyMzQ1Njc4OWFiY2RlZg==

Stricter OAuth conformance

Developers will need to follow the OAuth specification more closely when sending data to the SSO token endpoint. In short, to ensure that your application continues to work with the EVE SSO, you should follow the official EVE SSO guidelines.

Here are some important points to be aware of:

  • Native/mobile applications should use the PKCE flow as described here.
  • Web/server-side applications should use the authorization code flow as described here.
  • The v2/oauth/token endpoint will no longer accept data sent as a JSON payload or as a URL query. Data must be sent to the endpoint with Content-Type: application/x-www-form-urlencoded and encoded as such in accordance with Section 4.1.3 in OAuth RFC (6749).

Correct:

POST v2/oauth/token HTTP/1.1
Host: login.eveonline.com
Content-Type: application/x-www-form-urlencoded

client_id=someawesomeclient&grant_type=authorization_code&code=exampleauthorizationcode&code_verifier=codeverifier&redirect_uri=https%3A%2F%2Fmy3rdpartyapp%2Fauth%2Fcallback

Incorrect - Data sent as a JSON payload:

POST v2/oauth/token HTTP/1.1
Host: login.eveonline.com
Content-Type: application/json

{ "client_id":"someawesomeclient","grant_type":"authorization_code","code":"exampleauthorizationcode","code_verifier":"codeverifier","redirect_uri":"https://my3rdpartyapp/authþ/callback"}

Incorrect - data sent as a querystring:

POST v2/oauth/token?client_id=someawesomeclient&grant_type=authorization_code&code=exampleauthorizationcode&code_verifier=codeverifier&redirect_uri=https%3A%2F%2Fmy3rdpartyapp%2Fauth%2Fcallback HTTP/1.1
Host: login.eveonline.com

Use a standardized OAuth JWT validation library

Developers should use a standard OAuth JWT validation library to be prepared for upcoming changes in the SSO such as different JWT signature methods. Libraries exist for pretty much all programming languages out there. See JWT.IO for more information and an SDK for your programming language of choice!

Prepare for refresh token rotations

Developers have until now been able to safely assume that their stored refresh tokens will never change. This will no longer be the case, so developers should assume that the refresh token MIGHT change when it is refreshed and update it when needed.

Rate limits for failed requests

We'll be introducing a rate-limiter that will kick in when an application exceeds a certain threshold of failed requests. When it is triggered a 429 Too Many Requests response will be returned. This is to encourage developers to clear old/invalid refresh tokens from their databases as well as make sure their applications are well tested and conform to the OAuth specification. Applications that repeatedly trigger the rate-limiting and don't respect the Retry-After header might get their applications blocked by the EVE SSO for API Abuse and require a support ticket to be reopened

]]>
SSO Endpoint Deprecations sso-endpoint-deprecations-2
http://developers.eveonline.com/blog/article/v4-of-esi-public-character-information-to-be-removed-on-21.10.2021/ V4 of ESI public character information to be removed on 21.10.2021 2021-09-29T00:00:00Z 2021-09-29T00:00:00Z CCP Convict 2115597014 On 30 July 2020 in the ESI changelog we announced the deprecation of v4 version characters/{character_id}  ESI route on 15 September and subsequent removal on 21 September. Unfortunately we still see heavy traffic hitting that route which as of now has legacy status. Accordingly we've decided to postpone the (admittedly quite abrupt) removal. Its been rescheduled and now it will happen on 21st October 2021.  Please spread that information if you know any developer that might not follow changelog and/or Tweetfleet. ]]> V4 of ESI public character information to be removed on 21.10.2021 v4-of-esi-public-character-information-to-be-removed-on-21.10.2021 http://developers.eveonline.com/blog/article/sso-to-authenticated-calls/ ESI Step by Step - SSO to Authenticated Calls 2021-09-27T13:15:53Z 2021-09-27T13:15:53Z Team Tech Co 95870245 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


Deprecated Information

The information presented on this page is depreciated. Please see https://docs.esi.evetech.net/docs/sso/ for up-to-date information.


Last month, we talked about auto generating a client library for ESI using Swagger Codegen. This month we will expand on this by stepping through the SSO authentication flow and using that authentication to make a call to the ESI endpoint /characters/{character_id}/standings. Typically, one would set up a server to handle most of this process, but today we're going to do it manually with curl, to lay out the steps in the process. It is assumed the reader already knows what SSO is, if a refresher is needed look at our third party documentation on it here.

This blog assumes that you have worked through last month's blog on Swagger Codegen and a workspace with your work from this blog available. We're going to be adding a few lines of code to it at the end of this blog.

SSO Registration

Before the authentication flow can be implemented you must first go to EVE Online's developer site and login with your EVE Online account. Once you've logged in, click on Applications on the top toolbar. View and accept the developers license if prompted and on the next page click Create New Application. Once here there will be a page to fill in a Name and Description for your application. Because this blog will focus on getting access to the /characters/{character_id}/standings endpoint, name the application "Character Standings Checker" and set the description to "A simple tool designed to check the standings of characters.".

Next, for Connection Type, pick the radio button next to Authentication & API Access. This particular connection type is used for access to our third party APIs. Once this connection type is selected, a list of available scopes will be presented. Navigating to the /characters/{character_id}/standings swagger documentation and hovering over the red "!" sign will show that the esi-characters.read_standings.v1 scope is needed for this particular endpoint:

Find the esi-characters.read_standings.v1 scope in the Available Scopes List box and click on it. It will move over to the Requested Scopes List box when clicked on.

In the Callback URL field, enter the following URL: "http://localhost/oauth-callback". This is the URL that your users will be redirected to after login, and would typically be pointing at a webserver. As a security measure, the EVE SSO will only redirect users of your application to this callback URL.

Click the Create Application button to finish. Once the application has been created click on View Application and keep this tab open in your browser for later.

Executing the Authentication Flow

EVE Online's SSO uses what's known as the Authorization Code Grant OAuth 2 authorization flow (the RFC for this can be found here). A typical flow would function like this:

  • Log in to EVE Online's SSO using a URL with specific GET parameters.
  • After Logging in, be redirected to a running server at the callback URL that was defined in your SSO application (in this case http://localhost/oauth-callback). This redirection will include an authorization code supplied from EVE Online's SSO as a GET parameter.
  • Inside the server that handles the SSO callback, make a POST request back to EVE Online's SSO with the authorization code provided. This request will use HTTP basic authentication and will use an SSO application client ID as the user and the secret key as the password. The client ID and secret key can be found on the eve developers website, in the browser tab that we suggested to keep open at the end of the SSO Registration process above.
  • Finally, the response given back to the server will contain an access token that can be used to make an authenticated call to ESI.

In the next section, we're going to step through this by hand using curl. Along the way, you're going to see fields labelled {client id} and {client secret}. When you see these, substitute the client ID and client secret from the View Application page on https://developers.eveonline.com that you were viewing at the end of SSO Registration. Throughout this blog, we're going to use {} syntax to indicate "you should remove these brackets and everything between them, and replace it with a suitable value of your own in this place".

Create the log-on URL

The first step is to create a URL that users can follow to begin the logon process. This URL has the following structure:

https://{login server base url}/oauth/authorize?response_type={response type}&redirect_uri={redirect uri}&client_id={client id}&scope={scopes}&state={state}

{login server base url} is the url for the sso server. On tranquility, that url is "login.eveonline.com".

The following query parameters are mandatory:

  • Response type can be 'code' or 'token'. Setting response type to token will cause you to go down an alternate flow called the Implicit Grant, but we want an Authorization Code Grant, so we're going to set it to 'code'.

  • Redirect uri is where the user will be redirected after login. In other OAuth servers this might be a freeform field, but the EVE SSO will return an error if this does not match your clients designated callback, so we're going to set it to "http://localhost/oauth-callback" to match the value we entered during SSO registration.

  • Client ID is your application's client ID from the developers website.

The following query parameters are optional (although we need scopes in this case):

  • Scopes is a list of space-separated scopes that you wish to request for this specific authorization call. It can be a subset of the scopes that were chosen when creating the client. We're going to set it to "esi-characters.read_standings.v1".

  • State is a field that you can use to pass some amount of state information from your client to your server via the SSO. Anything entered in this parameter will be passed on to the redirect URI by the eve SSO if the login succeeds. We don't need it in this case.

Our final URL looks like this:

https://login.eveonline.com/oauth/authorize?response_type=code&redirect_uri=http://localhost/oauth-callback&client_id={client id}&scope=esi-characters.read_standings.v1

This is the URL that you would embed in your website with a "log into eve online" button.

Navigate to this URL in your browser, and log in.

Extract the Authorization Token from the redirected URL

Once you log into the SSO, the browser will be redirected to a URL that looks something like this:

http://localhost/oauth-callback?code=ckEZIa6JUOdoN6ijmqBI...qgpU-SmPsZ0

Typically, a server would receive this as a GET request from the browser, and automatically extract the code, but we're going to just copy it out of the URL manually. Copy the code query parameter value into a text editor.

The code in this URL is a one-use-only authorization code that can be exchanged with the SSO for an Access token and a refresh token.

Exchanging the authorization code for an access token.

To exchange the authorization code for an access token, we need to make a post request with curl.

The URL of the POST request has the following structure:

https://{login server base url}/oauth/token

In addition, the POST request needs headers and a body. We must send an Authorization header and a Content-Type header.

The Content-Type can be application/json or application/x-www-form-urlencoded. We're going to use JSON.

The Authorization header must be the word "Basic" followed by the base-64 encoded string of {client id}:{client_secret}. If the client id was "client_id" and the secret was "clientsecret1", the final Authorization header value would be "Basic Y2xpZW50X2lkOmNsaWVudHNlY3JldDE=". You can see how this was encoded by decoding the string on this website, but you should encode your own secret on your local computer, as it's unwise to input secrets in third party websites.

The body will be a json string, and looks like this:

    {
      "grant_type":"authorization_code",
      "code":"{the authorization code}"
    }

If we make this into a curl request, we end up with this:

curl -XPOST -H "Content-Type:application/json" -H "Authorization:Basic Y2xpZW50X2lkOmNsaWVudHNlY3JldDE=" -d '{"grant_type":"authorization_code", "code":"ckEZIa6JUOdoN6ijmqBI...qgpU-SmPsZ0"}' https://login.eveonline.com/oauth/token

Note that I've used a fake client ID and secret to generate the basic authentication code in this example. Remember to not use the code provided but instead replace it with one generated from your own client id and client secret. Execute the above call with your own authentication string, and the authorization code from the previous step. This call will succeed only once. Attempts to use the code a second time will result in the error "Authorization code not found".

You will get a response that looks like this:

{"access_token":"jZOzkRtA8B...LQJg2","token_type":"Bearer","expires_in":1199,"refresh_token":"RGuc...w1"}

The access token is the part we will need for the next step. Copy the access_token value given to you in the response into a text editor.

Note that access tokens are only valid for 20 minutes, after which you can re-run this step with the same headers and the following body to use the refresh token to get another access token at any time:

    {
      "grant_type":"refresh_token",
      "refresh_token":"{the refresh token}"
    }

Use the access token to get the character ID

We'll need a character ID for the next step. The SSO will provide this information from access tokens.

To get the character ID of a token, make a GET request to the following url:

https://{login server base url}/oauth/verify with the header Authorization: Bearer {access token}

In our case, it's going to be this:

curl -XGET -H 'Authorization: Bearer {access token from the previous step}' https://login.eveonline.com/oauth/verify

Which will return something similar to the following:

{"CharacterID":95465499,"CharacterName":"CCP Bartender","ExpiresOn":"2017-07-05T14:34:16.5857101","Scopes":"esi-characters.read_standings.v1","TokenType":"Character","CharacterOwnerHash":"lots_of_letters_and_numbers","IntellectualProperty":"EVE"}

Copy the character ID into a text editor.

Make an authenticated call.

In the previous blog we set up an environment with a python swagger client, and made an unauthenticated call to ESI. You'll need that environment for the next part of this blog, so if you don't have it set up, work through the codegen blog and return here once it's complete.

Once again, open a python shell in the appropriate virtualenv, and execute the following:

import swagger_client
from swagger_client.rest import ApiException
from swagger_client import Configuration

api = swagger_client.CharacterApi()
api.api_client.set_default_header('User-Agent', 'my-test-agent') # Set a relevant user agent so we know which software is actually using ESI
api.api_client.host = "https://esi.tech.ccp.is"
Configuration().access_token = "{access token}" # fill in your access token here
try:
    response = api.get_characters_character_id_standings({character id}) # fill in the character id here
    print(response)
except ApiException as e:
    print("Exception when calling CharacterApi->get_characters_character_id_standings: %s\n" % e)

And you're done! You've made an authenticated call to ESI.

XML Transition Project Update

Since the last blog, the following XML API endpoints have been ported to ESI:

category XML endpoint ESI parity
char    
. AccountBalance https://esi.tech.ccp.is/latest/#!/Wallet/get_characters_character_id_wallet
. CharacterSheet (not finished)
https://esi.tech.ccp.is/latest/#!/Character/get_characters_character_id
https://esi.tech.ccp.is/latest/#!/Clones/get_characters_character_id_clones
https://esi.tech.ccp.is/dev/#!/Skills/get_characters_character_id_skills
https://esi.tech.ccp.is/latest/#!/Skills/get_characters_character_id_attributes
. ContractBids https://esi.tech.ccp.is/latest/#!/Contracts/get_characters_character_id_contracts_contract_id_bids
. ContractItems https://esi.tech.ccp.is/latest/#!/Contracts/get_characters_character_id_contracts_contract_id_items
. Contracts https://esi.tech.ccp.is/latest/#!/Contracts/get_characters_character_id_contracts
. WalletJournal https://esi.tech.ccp.is/dev/#!/Wallet/get_characters_character_id_wallet_journal
. WalletTransactions https://esi.tech.ccp.is/latest/#!/Wallet/get_characters_character_id_wallet_transactions
eve    
. CertificateTree Deprecated
. RefTypes All expanded in relevant endpoints

Complete and up-to-date list can be found here.

]]>
SSO to Authenticated Calls sso-to-authenticated-calls
http://developers.eveonline.com/blog/article/esi-best-practices-using-underscore-routes/ ESI Best Practices: Generating Code With Underscore Routes 2021-03-11T17:22:16Z 2021-03-11T17:22:16Z CCP Bartender 95465499 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


This blog explains best practices for autogenerating language specific clients from the ESI swagger spec. When generating code from one of the named routes (e.g. https://esi.evetech.net/latest/swagger.json), you may have noticed the resulting client library uses /latest as the version in all its URL calls.

You can see why if you look at this fragment of the swagger spec from the above URL:

{
  "basePath": "/latest",
  "host": "esi.evetech.net",
  "info": {
    "description": "An OpenAPI for EVE Online",
    "title": "EVE Swagger Interface",
    "version": "0.4.4.dev4"
  },
  "paths": {
    "/alliances/": {
      "get": {
        "description": "List all active player alliances\n\n---\n\nAlternate route: `/v1/alliances/`\n\nAlternate route: `/legacy/alliances/`\n\nAlternate route: `/dev/alliances/`\n\n\n---\n\nThis route is cached for up to 3600 seconds",
        {...}
      }
    }
  }
}

The basePath for all URL's is set to /latest at the top level, and inherited by all paths. From the given example, the route /latest/alliances/ is assembled. That's a problem, because if a route currently part of /latest is bumped from v2 to v3, the autogenerated library will start calling the new /latest and could encounter errors. The best practice for stability is to pin the versions of all routes from /latest at the time code is tested and generated.

That capability is available in the form of "underscore routes". You can call https://esi.evetech.net/_latest/swagger.json and get a modified copy of the swagger spec, in which the basePath has been converted to the root /, and each route has been converted to its full versioned path:

{
  "basePath": "/",
  "host": "esi.evetech.net",
  "info": {
    "description": "An OpenAPI for EVE Online",
    "title": "EVE Swagger Interface",
    "version": "0.4.4.dev4"
  },
  "paths": {
    "/v1/alliances/": {
      "get": {
        "description": "List all active player alliances\n\n---\n\nThis route is cached for up to 3600 seconds",
        {...}
      }
    }
  }
}

From this example, the route /v1/alliances/ is assembled. This modified spec will never contain more than one version of a resource/verb pair, and you will find similar specs available at _dev and _legacy. When autogenerating code for ESI, it is best practice to use the underscore routes as a swagger source, to ensure the versions are pinned.

When using this technique, deprecation of routes is identified by watching for warning headers in the response. When a versioned route is moved from latest to legacy, it will start returning the following headers, even on an authentication failure:

{
  "server": "nginx",
  "date": "Mon, 08 May 2017 15:25:26 GMT",
  {...},
  "warning": "199 - This endpoint has been updated.",
  {...}
}

More details on this can be found in our blog about using versioned routes.

~CCP Bartender

]]>
ESI Best Practices Using Underscore Routes esi-best-practices-using-underscore-routes
http://developers.eveonline.com/blog/article/from-image-server-to-a-whole-new-image-service-1/ From Image Server to a Whole New Image Service 2021-03-11T16:13:32Z 2021-03-11T16:13:32Z CCP Convict 2115597014 As outlined by the previous blog, we’re upgrading the image server to an entirely new service that better fits the modern developer environment and needs. This means a friendlier, RESTful API, dynamic resizing, multi-tenancy support, and all these will be served through a global CDN.

A New Domain

Initially, we thought about sharing the same domain as the previous image server, but the change is way too subtle. So we will use a completely new domain for the new image service: images.evetech.net

Actually, the new service has already been deployed and running for a while, so go ahead and give it a try! It has some documentation with a very “minimalistic design” for now, which will likely be improved in the future.

The New APIs 
 

There are two main APIs in the new image service, we refer to them as “variations” and “image” here.

Variations

Different kinds of resources provide different variations of images. E.g. corporations have logos, characters have portraits, (EVE) types have both icons and renders. Variations for a resource are listed under resource_type/resource_id, for exmaple:

Images

Appending the variation to the path above, and the service will respond with actual images. Currently the service supports:

Query Parameters

Resizing and multi-tenancy are supported through query parameters: size and tenant, respectively. For exmaple:

Currently the service only supports a set of sizes that are powers of 2, i.e.: 32, 64, 128, 256, 512 and 1024. When size is not supplied explicitly, the service will use an arbitary size. So it’s usually better to query a specific size, especially when the target size is known ahead of time.

The service currently supports tranquility and singularity for tenants, and defaults to tranquility.

Migration Path

For new applications we recommend to start using the new image service immediately, with the new RESTful API. But for existing applications, we will try our best to provide an easier migration path, so that our developers that have poured a lot of effort into these applications will have enough time to try and test the new image service before deployment. The following will happen:

  • The new image service supports “old-style path” for now, e.g. https://images.evetech.net/Alliance/434243723_1024.png still works
  • We will change the DNS to point existing image server domains to the new service so that existing applications will continue to work for a while
    • bear in mind that this is nevertheless a network change, and the underlying network routes and even HTTP responses won’t be 100% the same. But as far as image content is concerned, we expect no change
  • For any request that uses the “old-style path”, the new image service will respond with a Warning header, indicating our intent to deprecate these in the future, just like how we deprecate legacy endpoints in ESI

Again, keep in mind this is only provided for smoothly transitioning developers of existing applications to the new image service. This will never be documented anywhere in the new image service, and we are planning to remove these support after a couple of months, for example when we’re happy with usage metrics on these “old-style path”, and the old domains will likely be deprecated at some point in the future as well.

Why the Upgrade

This change should (hopefully) be trivial for applications that use the current Image Server since it’s mostly just a URL path change - just shuffle a few things around in your printf.

However. this means a lot internally to many teams inside CCP and for EVE development. The current “pipeline” involves a lot of manual work which are time consuming and prone to error, the new pipeline will be fully automated and also integrated into the EVE build system. This means that asset changes will be reflected as they’re released - at most a couple of hours due to CDN caching - and developers will be immediately notified on any failure at build time.

On top of the pipeline change, the multi-tenancy support means it’s a hell of a lot easier to support images for local servers we EVE developers use during day-to-day work. This bridges another gap between what a developer sees locally and what EVE players see after changes are released, so there will be less chance for bad changes to reach production servers unnoticed.

As famously said, a small step for you, but a big step for us!

And the IEC?

As mentioned in the previous blog, we will provide a final package for IEC and stop updating them. This plan is unchanged, at least in the forseeable future. The blocker for us to support it on the new image service boils down to discovery and addressability. But we will be happy to work with the community to seek a solution if there is enough interest.

Thanks for All the Fish

Before closing out, remember that all usage of developer resources need to be in compliance with our license.

And now, let’s say goodbye to the good old Image Server!

]]>
From Image Server to a Whole New Image Service from-image-server-to-a-whole-new-image-service-1
http://developers.eveonline.com/blog/article/removal-of-scopes-from-sso/ REMOVAL of scopes from SSO 2020-03-05T00:00:00Z 2020-03-05T00:00:00Z CCP Convict 2115597014 On 8 April 2020, we will be making the following change:
 
REMOVAL of scopes from SSO
  • esi-corporations.read_outposts.v1;
  • esi-corporations.write_structures.v1.
These scopes have not served any purpose for a long time now but we see that some applications still request them. If those applications aren't updated they will break after scopes removal.
]]>
REMOVAL of scopes from SSO removal-of-scopes-from-sso
http://developers.eveonline.com/blog/article/removing-datasource-singularity/ Removing Datasource Singularity 2020-01-13T00:00:00Z 2020-01-13T00:00:00Z Team Tech Co 95870245 Removing Datasource Singularity

In our efforts to increase ESI production stability, and limit our maintenance overhead, we will be removing Singularity as an available datasource for ESI production.

This action is planned to take effect during downtime tomorrow, 14th of January 2020.

-- Team Tech Co

Want to be part of the ESI conversation? Join the #esi slack channel and become a voice in our community.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

]]>
Removing Datasource Singularity removing-datasource-singularity
http://developers.eveonline.com/blog/article/temporary-blackout-of-esi-bookmarks/ Temporary blackout of ESI-bookmarks 2019-11-25T00:00:00Z 2019-11-25T00:00:00Z Team Tech Co 95870245 Temporary blackout of esi-bookmarks

With the coming changes to our bookmark systems, and how internal schedules have lined up, we will be temporarily shutting down esi-bookmarks routes to re-implement them for the new system. This will result in empty list returns to reduce the impact on third party developers.

The affected routes are:

  • /characters/{character_id}/bookmarks/
  • /characters/{character_id}/bookmarks/folders/
  • /corporations/{corporation_id}/bookmarks/
  • /corporations/{corporation_id}/bookmarks/folders/

The new character routes will be deployed as soon as possible, with a new data schema, so that pilots can access all their new bookmarks, as well as their folders. The corporation endpoints will remain closed as the new bookmarks system does not call for their reimplementation.

We apologize for any inconvenience caused by this temporary blackout and hope to have services restored in the near future.

o7

-- Team Tech Co

Want to be part of the ESI conversation? Join the #esi slack channel and become a voice in our community.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

]]>
Temporary blackout of ESI-bookmarks temporary-blackout-of-esi-bookmarks
http://developers.eveonline.com/blog/article/upcoming-changes-to-the-image-export-collection/ Upcoming changes to the Image Export Collection 2019-07-10T00:00:00Z 2019-07-10T00:00:00Z Team Tech Co 95870245 Greetings developers!

 

As was mentioned at EVESterdam 2019 at the start of the third party developers panel, we're looking into deprecating the IEC and upgrading the image server to better support cases that people were using the IEC for. This will remove the need for a lot of work to be done every release by team TriLambda, as the IEC generation pipeline is very old and radically different from EVE's modern graphics processes.

 

Along with moving images from the IEC to the image server, the image server itself will be getting a facelift. In short:

 

1. The new image server will be (but has not yet been) deployed to a new url (image->images), and support multitenancy though a query param (defaults to TQ), e.g.:

 

    

 

2. The Asset API for the image server will be formalized better, and will broadly take the form of 

 

some types have special variations, e.g. blueprints have bp and bpc. A query to /{category}/{id}?tenant={tenant} will return a json document listing available variations.

 

3. Requests to the Asset API will be redirected to CDN's across the globe, providing much faster response times.

 

4. The first iteration of the new image server will not provide scaling and simply return the largest size available for developers to scale as they see fit, but we welcome feedback on this and if enough cases come up from the dev community we will seriously consider how to provide such functionalities

 

When it comes to the IEC, the variants path is the missing piece that was forcing a lot of developers to fall back to the IEC. Most types were already available, but the variants for things like blueprints and relics were not.

The future of the IEC is currently planned to follow this sequence:

 

  1. Deploy the new image server side-by-side with the old one. 
  2. Stop updating the IEC
  3. Release a package of UI icons on the developer resources webpage

 

As you can see in step 3, we will be moving the UI icons into a separate zip. The UI icons are currently not addressed or ordered in any useful way, and a lot of work would need to be put in by the UI team to organize them into a long-term structure and update the client to use that new structure. Fortunately, the UI icons that are widely used by third party developers do not change much, and so this UI icons zip represents a medium-term solution to this problem. We will be generating this package once and it will then be the source of UI icons for third party developers until the refactoring of UI icons can happen.

 

If you are affected by these changes and would like to make specific requests or give us feedback, we would like to hear from you. We would also like to hear from third party developers who are heavy users of the IEC if they think there is a specific subset of the UI icons that would be disproportionately useful. Without making any promises, there has been some discussion of possibly putting a small subset of the UI icons on the image server, and we would like to know more about how people use the UI icons.

 

If you have an active interest in the IEC and have questions or comments, please submit your feedback in the feedback thread here.

]]>
Upcoming changes to the Image Export Collection upcoming-changes-to-the-image-export-collection
http://developers.eveonline.com/blog/article/technical-note-dont-forget-to-url-encode-your-query-params/ Technical note: don't forget to URL encode your query params! 2019-07-10T00:00:00Z 2019-07-10T00:00:00Z CCP Dopamine 2114766462 Greetings space developers!

Teams Tech Co and BBS have been looking into a small flutter of errors that we started seeing this morning after a new deployment of the SSO. During this deployment, we changed a few details about the structure of SSOv1 access tokens and there's been some interesting fallout.

The short version is that prior to today, v1 access tokens have always consisted of characters that look the same regardless of whether the token is URL encoded or not. After today, that is no longer the case. The tokens are now also slightly longer than they were yesterday.

We are seeing a small number of applications that are using ESI's query parameter token feature to make their requests but are not URL encoding their tokens. This results in tokens that are invalid being transmitted to ESI, which results in a failed authorization. We are also seeing some applications are truncating the now slightly longer tokens before transmission, which results in similar authorization errors.

It's important to always URL encode your query parameters before dispatching them to ensure that they do not become malformed during transmission. In the short term, we have elected to monkey-patch the sso-backend to be tolerant of the specific issue with + symbols being converted into spaces that several applications are seeing today, but we do not guarantee this convenience beyond Nov 1st, so please implement URL encoding prior to this date.

Unfortunately, there is nothing we can do to help applications that are truncating their tokens before transmission. If your application is doing this you'll need to update it to remove the token length assumption ASAP. We suspect that there are a few DB's around with VARCHAR(x) assumptions encoded into their token table schema.

Carry on!

 

Tech Co & BBS

]]>
Technical note: don't forget to URL encode your query params! technical-note-dont-forget-to-url-encode-your-query-params
http://developers.eveonline.com/blog/article/using-esi-with-google-sheets/ Using ESI With Google Sheets 2019-05-02T10:15:42Z 2019-05-02T10:15:42Z Team Tech Co 95870245 This guide was written for an older version of GESI.  For the latest instructions visit the setup instructions on the GESI GitHub repo.

This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018.


Come May 8th, the death of the XML API will also mean the death of many EVE based spreadsheets. Luckily, with the help of community member Blacksmoke16's GESI, the process for making an ESI based Google Sheet is much more simple than it has ever been. This blog will walk through how to make a Google Sheet to track your character's in-game assets using authenticated calls with GESI without needing to write any code yourself.

Creating a Google sheet and adding GESI

  1. Go to sheets.google.com and create a new blank spreadsheet.
  2. On the top bar of your spreadsheet click on Tools -> Script Editor.

    Tools -> Script Editor

This will open a new tab to an instance of https://script.google.com that is tied to your spreadsheet.

  1. Navigate to https://github.com/Blacksmoke16/GESI in a different tab and copy the contents of GESI.gs, endpoints.gs, and functions.gs into three different script files in your Google Scripts project.

Copy code demo
Click here for full screen

  1. Next, you will have to make an application for EVE's Single Sign On (SSO). ESI uses an authentication method called OAuth2. You don't need to know exactly how it works, but you do need what's called a callback URL for EVE's servers to contact during the authentication process. When using Google Scripts, the callback URL will look like https://script.google.com/macros/d/{SCRIPT_ID}/usercallback where {SCRIPT_ID} will be replaced with your Google Scripts project's "Script ID". To get this ID, navigate to File -> Project properties:

File -> Project properties

This will open a dialog box with some info about the Google Scripts project, look for "Script ID":

script_id

In another tab, open up https://developers.eveonline.com/ and log into your EVE account. Once logged in:

  • Click "Manage Applications".
  • On the next page, click "Create New Application".
  • On the next page fill in the "Name" and "Description" box with whatever you would like.
  • Under "Connection Type", choose "Authentication & API Access" which should make a new section called "Permissions" show up.
  • In the "Permissions" section, find the scope esi-assets.read_assets.v1 and click on it. Since all you are going to be doing is looking at your character's assets this is the only scope you want to give permission for. Once you know other authenticated routes you want to use you can come back and add more permissions. The scope needed for a particular endpoint can be found in ESI's swagger spec found at https://esi.tech.ccp.is/_latest/swagger.json Note: if you're using Chrome you can use an extension like JSONView to help make the contents of this link more human readable.
  • In the callback URL text box, enter https://script.google.com/macros/d/{SCRIPT_ID}/usercallback. Go back to the tab with your script ID, copy it, and replace {SCRIPT_ID} in the previous URL with your actual script ID. Using the script ID shown in the picture above, the full callback URL would be https://script.google.com/macros/d/1sH0cjPy7DuryOXRSZ5ZaCEzPB6yQGcRtKiFfqpIjB-LzS01M7sDC5G7w/usercallback
  • Click "Create Application"
  • On the next page, click "View Application" next to the name of the application you just made. You will have to keep this open because you will need the "Client ID" and "Secret Key" for the next step.

View Application

Modifying GESI code

In your Google Scripts project you will need to make a few modifications to GESI.gs:

  • Find the following lines:

    CLIENT_ID = 'YOUR_CLIENT_ID';
    CLIENT_SECRET = 'YOUR_CLIENT_SECRET';
    

    and replace YOUR_CLIENT_ID with your SSO application's client ID and replace YOUR_CLIENT_SECRET with your SSO application's secret key.

    It is important to remember that if anyone gets your client ID and secret key that they can make calls on behalf of your character. In this case, anyone who has access to your Google Sheet will be able to see this secret key so make sure to not make your Google Sheet public. This is also why it's important to not choose all scopes that are available when making your SSO application.

    • Find the line:
      MAIN_CHARACTER = 'YOUR_MAIN_CHARACTER_NAME';
      

    and replace YOUR_MAIN_CHARACTER_NAME with the name of your main EVE Online character.

  • Find the lines that look like:

SCOPES = [
    "esi-alliances.read_contacts.v1",
    "esi-assets.read_assets.v1",
    "esi-assets.read_corporation_assets.v1",
    "esi-bookmarks.read_character_bookmarks.v1",
    "esi-bookmarks.read_corporation_bookmarks.v1",
    ... // This denotes a continuation of this array
];

and remove all scopes except for esi-assets.read_assets.v1 to match your SSO application. SCOPES should look like the following when you are finished:

SCOPES = [
    "esi-assets.read_assets.v1",
];

You will need to add any scopes here that you add to your SSO application as well.

When you are finished with the previous three steps the top of GESI.gs should look similar to:

// Google ESI (GESI)
//
// /u/blacksmoke16 @ Reddit
// @Blacksmoke16#0016 @ Discord
APP_VERSION = '5.0.1';
BASE_URL = 'https://esi.tech.ccp.is'

// Setup variables used throughout script
// From your dev app https://developers.eveonline.com/applications
CLIENT_ID = 'f9581622f6134a469c26cbaffef079a7';
CLIENT_SECRET = 'OmyvILsIcfzKKcjsAOo5Ce1CTjaWexNeEiYR881B';

// Name of your 'main' character to use when a function is called without a character name as a param
MAIN_CHARACTER = 'CCP Zoetrope';

// List of scopes to request
SCOPES = [
    "esi-assets.read_assets.v1",
];

... // This denotes continuation of the file

Make sure to save GESI.gs when you're done.

Authorizing your spreadsheet

If you followed the previous steps correctly you should now be able to authorize your EVE account inside your spreadsheet!

Navigate back to the tab with the spreadsheet in it and reload the page. Upon reload and after waiting a bit you should see the option GESI appear on the toolbar on the top of the page. Click GESI -> Authorize Sheet which should open up an alert saying "Authorization Required". You will need to authorize the scripts that were made in Google Scripts to work for this spreadsheet.

discover GESI
Click here for full screen

Clicking "Continue" in the alert will open up a new browser window with a warning saying "This app isn't verified". This is because the scripts you are using have not been uploaded as official Add-ons for Google Sheets so it has to make sure you can trust the scripts you are about to run. Continue on by clicking "Advanced" and then clicking "Go to ESI Demo (unsafe)". The next page will list all the things that Google Scripts is asking permission for. This will all look similar to the following gif:

Google auth

Click "Allow" which will bring you back to your spreadsheet and display a dialog box with a link titled "Authorize with EVE SSO".

Authorize EVE SSO

Once you click the link you will be routed to the EVE Login page where you will be prompted to log in. Once logged in, choose a character from the drop down and then click "Authorize". If everything was successful you will be routed to a blank page with the words "Thank you for using GESI {your character name}" where {your character name} will be replaced with the actual name of your character.

GESI confirmation screen

You can close this tab if everything was successful and go back to your spreadsheet where you can now work with your assets!

Getting your character's assets in your spreadsheet

The route you will be using to get your character's assets is https://esi.tech.ccp.is/v3/characters/{character_id}/assets/. You can navigate to https://esi.tech.ccp.is/ui/ to browse through all endpoints available in ESI and you can click here to see the character assets endpoint specifically.

Now that you know what route you want to use go back to your Google Scripts (Tools -> Script editor) and open up endpoints.gs. In there, press Ctrl+F to bring up the search box and search for "/characters/{character_id}/assets/". This will bring you to a JSON object named characters_character_assets which will be the name of the function you will use in your spreadsheet.

Find function name
Click here for a full screen

Now that you know the name of the function to use in your spreadsheet you can head back plug it in.

Go to cell A1, type =characters_character_assets() and press Enter. After a bit of loading, you should see your spreadsheet be populated with data about your character's assets.

ESI only returns ID's for a lot of endpoints so you will see a list of type IDs representing your assets. You will need to take these type IDs and plug them into another endpoint to get details about each one. The endpoint you will need to use to get the name and description for a type ID is /universe/types/{type_id}/. You can use the same method explained above to find the name of the function you will need to use in your spreadsheet for this endpoint. After looking at endpoints.gs you will find that the function name is universe_types_type.

Go to cell H1 (or whichever column is blank after your asset's data) and enter the text "name". In cell H2, copy and paste the following function:

=HLOOKUP("name", universe_types_type("en-us", INDIRECT(ADDRESS(ROW(), 1)), True), 2, FALSE)

and press enter. It is left as an exercise for the reader to figure out how this function works exactly.

After hitting enter you should see the name of the type_id at the beginning of the row show up. Hover over the bottom right of cell H2 until drag down until you have covered all the rows of your asset data to apply this function to every row.

cell box example

Follow the same process used to get the "name" of the type_id to get the "description" using the following function:

=HLOOKUP("description", universe_types_type("en-us", INDIRECT(ADDRESS(ROW(), 1)), True), 2, FALSE)

Adding data to spreadsheet
Click here for full screen

You should now have a spreadsheet that allows you to keep track of your character's assets!

Conclusion

Spreadsheets can be a fun and familiar way to explore ESI's data and thanks to the pioneering efforts of both Steve "Fuzzysteve" Ronuken and Blacksmoke16 it is now much easier to do!

Keep in mind that Google has a limit on how many requests you can make from a spreadsheet per day. If you need something more complicated than is allowable then pick a programming language and try and build your own application!

As always you can chat with us and our helpful community on our #esi slack channel

o7

-- Team Tech Co.

Appendix

The following routes have been added or updated in ESI since the previous blog:

New versions of existing routes:

Bumped routes:

Removed routes:

  • PUT /v1/corporations/{corporation_id}/structures/{structure_id}/

Get a sneak peek at what's coming to ESI by watching our deployment timeline.

New to ESI? Check out our ESI quick reference.

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST.

]]>
Using ESI With Google Sheets using-esi-with-google-sheets
http://developers.eveonline.com/blog/article/third-party-dev-roundtable-at-evesterdam/ Third party dev roundtable at Evesterdam 2019-03-18T16:53:16Z 2019-03-18T16:53:16Z CCP Guard 90610935 Greetings spaceship developers!

As you may already know, CCP is taking Fanfest on the road this year with the EVE Invasion World Tour. On March 23-24 CCP will be joining forces with Evesterdam to bring you beer, spaceships, presentations and more in the Compagnietheater at the heart of Amsterdam.

We will be hosting a third-party development roundtable with the theme:  "What do you need from the API to make your dream project a reality?"

CCP Bartender and CCP Fozzie will be at the roundtable to discuss both the technical, and gameplay implications the API has. We want to hear what you have done, what you are currently working on, and what you might do in the future with the EVE API.

UPDATE: The roundtable will be streamed live at 15:30 UTC (EVE time) on Saturday, March 23rd and we'll be taking questions both from the room and from viewers on Twitch.

We hope to hear from experienced developers but also from anyone who's ever been interested in getting into third party development for EVE, regardless of experience. 

Fly safe!

Team Tech Co

]]>
Third party dev roundtable at Evesterdam third-party-dev-roundtable-at-evesterdam
http://developers.eveonline.com/blog/article/removal-of-redirect-from-.tech.ccp.is-on-jan-7th/ Removal of redirect from .tech.ccp.is on Jan 7th 2018-12-20T10:13:32Z 2018-12-20T10:13:32Z Team Tech Co 95870245 Greetings Developers!

Yesterday was a day of some disruption to ESI, for which Tech Co apologises. During the process of resolving yesterday's issues it was necessary to discard the DNS entries for esi.tech.ccp.is. As a result, ESI was for approximately 20 hours only available on https://esi.evetech.net/

The esi.evetech.net URL has been the standard URL for ESI for approximately 6 months now, and the redirect from tech.ccp.is has been in place as a courtesy to allow developers some time for transition. It seems that Tech Co's communication on this matter has been below our desired standard and although most developers made the switch some time ago, not all developers were aware. For that too, we apologise.

For reference, the matter has been discussed on the #esi tweetfleet slack and was alluded to in a dev blog but was only explicitly stated in a tweet, which we do not consider a core communication tool.

Our initial decision yesterday afternoon was to not reinstate the redirect. Our reason for this decision was twofold: There had been a very reasonable amount of time to make this switch, and we did not wish to muddy the waters on this matter yet further. However, after internal discussion about the amount of frustration this decision caused and particularly considering we're coming in to the weekend before Christmas, we've decided to reverse that decision.

The tech.ccp.is redirect has been reinstated and will remain until Monday 7th of January. Third party developers, please consider this formal notice that you must update your esi URL's from esi.tech.ccp.is to esi.evetech.net before that date.

Carry on!

- Team Tech Co.

]]>
Removal of redirect from .tech.ccp.is on Jan 7th removal-of-redirect-from-.tech.ccp.is-on-jan-7th
http://developers.eveonline.com/blog/article/changes-coming-to-the-eve-online-imageset/ Changes Coming To The EVE Online ImageSet 2018-09-10T13:59:51Z 2018-09-10T13:59:51Z Team TriLambda 95835963 For every new EVE Online release, the Audio and Graphics Group delivers the ImageSet: a collection of graphic files available for use by the community.

The ImageSet changes as assets, icons, and brackets are modified, as well as when the tools we use to create those images improve. Through the August release, incorporating those improvements has not required any changes to nomenclature of existing files within the ImageSet itself.

This will change with the September release.

Due to a series of fixes, the ImageSet Bracket folder will be dramatically modified to include brackets in use in the client while older unused brackets will be removed. Almost all previously existing icons will have new names, the number of icons in the folder will increase from 19 to 222, and we will begin to provide ship bracket icons in both 16x16 and 32x32 pixel dimensions.

We advise the community and especially those maintaining resources which make use of the ImageSet to prepare updates to any systems that depend upon the current files in the Brackets folder.

Here's a list of the old and new images in the Brackets folder, for reference. Bracket icon count went up from 19 to 222.

List of Original Bracket Images:

  •                 constellation.png
  •                 NPCShipIndicator.png
  •                 region.png
  •                 shipBattlecruiser.png
  •                 shipBattleship.png
  •                 shipCapital.png
  •                 shipCapsule.png
  •                 shipCruiser.png
  •                 shipDestroyer.png
  •                 shipFrigate.png
  •                 shipIndustrial.png
  •                 shipIndustrialCapital.png
  •                 shipIndustrialCommand.png
  •                 shipMiningFrigate.png
  •                 shipRookie.png
  •                 shipShuttle.png
  •                 shipSuperCapital.png
  •                 shipTitan.png
  •                 solarSystem.png

 

List of New Bracket Images:

  •                 agentInSpace.png
  •                 assemblyArray.png
  •                 asteroidBelt.png
  •                 asteroidLarge.png
  •                 asteroidMedium.png
  •                 asteroidSmall.png
  •                 auditLogSecureContainer.png
  •                 battery.png
  •                 batteryElectronicWarfare.png
  •                 batteryEnergyNeutralizing.png
  •                 batterySensorDampening.png
  •                 batteryStasisWebification.png
  •                 batteryWarpScrambling.png
  •                 battleCruiser_16.png
  •                 battleCruiser_32.png
  •                 battleship_16.png
  •                 battleship_32.png
  •                 beacon.png
  •                 billboard.png
  •                 bomb.png
  •                 capsule_16.png
  •                 capsule_32.png
  •                 capturePointTower.png
  •                 carrier_16.png
  •                 carrier_32.png
  •                 citadelExtraLarge.png
  •                 citadelLarge.png
  •                 citadelMedium.png
  •                 combatSite_16.png
  •                 commandNode.png
  •                 compressionArray.png
  •                 constellation.png
  •                 constructionPlatform.png
  •                 containerCargo.png
  •                 containerCargoNPC.png
  •                 containerCargo_20x20.png
  •                 containerFreight.png
  •                 containerMission.png
  •                 containerSpew.png
  •                 controlBunker.png
  •                 controlTower.png
  •                 corpHangarArray.png
  •                 couplingArray.png
  •                 cruiser_16.png
  •                 cruiser_32.png
  •                 customsOffice.png
  •                 customsOfficeNPC.png
  •                 cynosuralField.png
  •                 cynosuralGeneratorArray.png
  •                 cynosuralSystemJammer.png
  •                 data_Site_16.png
  •                 deadspaceOverseersBelongings.png
  •                 deadspaceOverseersSentry.png
  •                 destroyer_16.png
  •                 destroyer_32.png
  •                 destructibleSentry.png
  •                 destructibleStationServices.png
  •                 dreadnought_16.png
  •                 dreadnought_32.png
  •                 droneAttack_16.png
  •                 droneEW_16.png
  •                 droneEW_L_16.png
  •                 droneEW_M_16.png
  •                 droneEW_S_16.png
  •                 droneFighterBomber_16.png
  •                 droneFighter_16.png
  •                 droneHeavyAttack_16.png
  •                 droneLightScout_16.png
  •                 droneLogistics_16.png
  •                 droneLogistics_L_16.png
  •                 droneLogistics_M_16.png
  •                 droneLogistics_S_16.png
  •                 droneMediumScout_16.png
  •                 droneMining_16.png
  •                 droneMining_M_16.png
  •                 droneMining_S_16.png
  •                 droneSalvaging_16.png
  •                 droneSalvaging_S_16.png
  •                 droneSentry_16.png
  •                 encounterSurveillanceSystem.png
  •                 engineeringComplexExtraLarge.png
  •                 engineeringComplexExtraLargeNPC.png
  •                 engineeringComplexLarge.png
  •                 engineeringComplexMedium.png
  •                 entity.png
  •                 fighterSquadH_16.png
  •                 fighterSquadM_16.png
  •                 fighterSquad_16.png
  •                 forceAuxiliary_16.png
  •                 forceAuxiliary_32.png
  •                 forceFieldArray.png
  •                 freighter_16.png
  •                 freighter_32.png
  •                 frigate_16.png
  •                 frigate_32.png
  •                 frozenCorpse.png
  •                 harvestableCloud.png
  •                 iceAsteroidLarge.png
  •                 iceAsteroidMedium.png
  •                 iceAsteroidSmall.png
  •                 iceField.png
  •                 industrialCommand_16.png
  •                 industrialCommand_32.png
  •                 industrial_16.png
  •                 industrial_32.png
  •                 infrastructureHub.png
  •                 jumpPortalArray.png
  •                 largeCollidableStructure.png
  •                 miningBarge_16.png
  •                 miningBarge_32.png
  •                 miningFrigate_16.png
  •                 miningFrigate_32.png
  •                 mobileCynosuralInhibitor.png
  •                 mobileJumpDisruptor.png
  •                 mobileLab.png
  •                 mobileMicroJumpUnit.png
  •                 mobileReactor.png
  •                 mobileReprocessingArray.png
  •                 mobileScanInhibitor.png
  •                 mobileSentry.png
  •                 mobileSentryHybrid.png
  •                 mobileSentryLaser.png
  •                 mobileSentryMissile.png
  •                 mobileSentryProjectile.png
  •                 mobileSiphonUnit.png
  •                 mobileStorage.png
  •                 mobileTractorUnit.png
  •                 mobileWarpDisruptor.png
  •                 moon.png
  •                 moonAsteroidJackpot.png
  •                 moonAsteroidRegular.png
  •                 moonMining.png
  •                 npcbattleCruiser_16.png
  •                 npcbattleCruiser_32.png
  •                 npcbattleship_16.png
  •                 npcbattleship_32.png
  •                 npccapsule_16.png
  •                 npccapsule_32.png
  •                 npccarrier_16.png
  •                 npccarrier_32.png
  •                 npccruiser_16.png
  •                 npccruiser_32.png
  •                 npcdestroyer_16.png
  •                 npcdestroyer_32.png
  •                 npcdreadnought_16.png
  •                 npcdreadnought_32.png
  •                 npcdroneAttack_16.png
  •                 npcdroneEW_16.png
  •                 npcdroneFighterBomber_16.png
  •                 npcdroneFighter_16.png
  •                 npcdroneLogistics_16.png
  •                 npcdroneMining_16.png
  •                 npcdroneSalvaging_16.png
  •                 npcdroneSentry_16.png
  •                 npcfighterSquad_16.png
  •                 npcforceAuxiliary_16.png
  •                 npcforceAuxiliary_32.png
  •                 npcfreighter_16.png
  •                 npcfreighter_32.png
  •                 npcfrigate_16.png
  •                 npcindustrialCommand_16.png
  •                 npcindustrialCommand_32.png
  •                 npcindustrial_16.png
  •                 npcindustrial_32.png
  •                 npcminingBarge_16.png
  •                 npcminingBarge_32.png
  •                 npcminingFrigate_16.png
  •                 npcminingFrigate_32.png
  •                 npcrookie_16.png
  •                 npcrookie_32.png
  •                 npcshuttle_16.png
  •                 npcshuttle_32.png
  •                 npcsuperCarrier_16.png
  •                 npcsuperCarrier_32.png
  •                 npctitan_16.png
  •                 npctitan_32.png
  •                 npfrigate_32.png
  •                 ore_Site_16.png
  •                 ownShip.png
  •                 personalHangarArray.png
  •                 planet.png
  •                 protectiveSentry.png
  •                 refineryLarge.png
  •                 refineryMedium.png
  •                 region.png
  •                 relic_Site_16.png
  •                 rookie_16.png
  •                 rookie_32.png
  •                 scannerProbe.png
  •                 scatterContainerBeingHacked.png
  •                 scatterContainerEmpty.png
  •                 scatterContainerOpened.png
  •                 scatterContainerUnopened.png
  •                 secureCargoContainer.png
  •                 self_16.png
  •                 sentry.png
  •                 shieldHardeningArray.png
  •                 ship.png
  •                 shipMaintenanceArray.png
  •                 shipMaintenanceArrayXL.png
  •                 shuttle_16.png
  •                 shuttle_32.png
  •                 silo.png
  •                 solarSystem.png
  •                 sovereigntyBlockadeUnit.png
  •                 spawnContainer.png
  •                 stargate.png
  •                 station.png
  •                 stationUpgradePlatform.png
  •                 structure.png
  •                 sun.png
  •                 supercarrier_16.png
  •                 superCarrier_32.png
  •                 territorialClaimUnit.png
  •                 titan_16.png
  •                 titan_32.png
  •                 warpGate.png
  •                 wormhole.png
  •                 wreck.png
  •                 wreckLooted.png
  •                 wreckLootedNPC.png
  •                 wreckNPC.png
]]>
Changes Coming To The EVE Online ImageSet changes-coming-to-the-eve-online-imageset
http://developers.eveonline.com/blog/article/the-esi-api-is-a-shared-resource-do-not-abuse-it/ The ESI API is a shared resource, do not abuse it 2018-06-19T16:01:22Z 2018-06-19T16:01:22Z Team Tech Co 95870245 Greetings developers.

As you all know, the ESI API is a shared resource among all of New Eden's citizens, and we work closely with developers to foster good practices and ensure everyone has equal access to it. Team Tech Co has for some time now been monitoring the usage of the /search/ routes, and we are sad to report that several developers are placing disproportionate load on the search service in order to achieve goals that it was never intended to serve.

The goal of the search endpoint is to allow developers to upstream the searching for and autocompletion of the names of various types of entities in the eve universe. This saves third party developers the trouble of implementing their own search whilst also providing a more comprehensive search experience, and makes life better for everyone. Unfortunately, some developers have for some time now been using it as a discovery endpoint for various things by dumping extremely large numbers of carefully structured requests into it. 

When this behavior first started, it was done carefully and relatively well optimized, and we felt we could keep an eye on it and see how the situation developed. It has developed poorly. Since then, the developers doing this have become considerably messier in their use and the situation can no longer be tolerated.

This kind of behavior is exactly why most API's have rate limits, but Team Tech Co. believes strongly that the ESI API is better without a rate limit. In the past we have relied on the error limiter to curb this kind of behavior, but the error limiter does not suit defending against this specific case.

From this point forwards, using the search endpoints as a discovery mechanism will be considered abuse of the API, and will be met with bans. There will be a grace period of 1 week from the release of this blog for people to turn off their scrapers. People who increase the rate of scraping during this week will be banned from the API.

We ask you to respect this ruling. If developers are unable to treat the ESI resources with respect, it may be necessary to reconsider the policy of not using rate limits.

Thanks for your assistance in this matter,

Team Tech Co.

]]>
The ESI API is a shared resource, do not abuse it the-esi-api-is-a-shared-resource-do-not-abuse-it
http://developers.eveonline.com/blog/article/updated-esi-quick-reference-for-new-developers/ Updated ESI Quick Reference For New Developers 2018-06-19T15:59:27Z 2018-06-19T15:59:27Z Team Tech Co 95870245 ESI Quick Reference For New Developers

This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018.


With the recent announcement that ESI has now reached parity with XML API and CREST there will undoubtedly be newcomers to ESI (welcome!). A lot of important information about ESI is strewn about in various blogs and may be overwhelming to newcomers. This blog will act as a quick reference for the most important things every new ESI developer should know listed all in one document.

What is ESI?

A RESTful API for EVE built using the OpenAPI 2.0 specification. Keep in mind OpenAPI used to be referred to as Swagger so any previous documentation will use that term.

Will ESI have account level info?

ESI is a character oriented API and will never have account level endpoints.

Is ESI rate limited?

No, but there is an error limit. As explained here and here if you generate 100 errors in a 60 second time window your applications will start receiving the HTTP status code 420 returned for every request it makes until the 60 second time window is done. For more info on how to watch for this read the previously linked blogs.

How does authentication work with ESI?

ESI authenticated routes can be accessed through EVE's SSO (single sign on). This previous blog takes you through a step by step process for authenticated calls.

Where is the documentation for ESI?

ESI's documentation sits in its code and is automatically generated using the OpenAPI specification. You can find a list of all available routes on the ESI Swagger UI found at https://esi.evetech.net/. You can expand a given endpoint and see exactly what model it returns and even try out routes directly through the UI.

Is there any documentation showing ESI equivalent endpoints to XML API and CREST?

Why yes there is!

How does versioning work in ESI?

All routes released on ESI have numbered versions (e.g. v1, v2, v3) and are tagged with one of three labels: dev, latest and legacy. The following diagram is a good top level overview of the lifecycle of a route:

lifecycle of an ESI route

The short version: always use the versioned routes in your software, never use labels. Labels can change, versions do not.

How do you know if the versioned route you are using is deprecated?

Watch for the Warning HTTP header in all your calls, you will see something similar to Warning: 299 - This endpoint is deprecated. That means next time there is a version bump on that route then the version you are using will start 404'ing.

How do you automatically generate a client for ESI using OpenAPI?

You must use the JSON version of ESI's OpenAPI specification when generating a client. You can get the full specification for all routes categorized under each of the three labels legacy, dev, and latest at https://esi.evetech.net/_<label>/swagger.json where <label> should be replaced with one of the three labels used in ESI. Notice the "_" before the label, this will make it so all the routes will have the version hardcoded into the URL rather than the label.

This is easier seen than explained, so to see what the previous paragraph means first go to https://esi.evetech.net/latest/swagger.json and then visit https://esi.evetech.net/_latest/swagger.json. Note the differences in URLs for the routes. Always use the "undescore" route.

A previous blog post gives detailed instructions for exactly how to generate a client library. Make sure to look at this blog if you run into problems.

How can you see differences between version's specifications?

We recently released a diff UI that will show you differences per route between two different versions.

How do I report a bug or ask for a new feature in ESI?

Make a new issue in our public GitHub repository.

How do I report a bug or ask for a new feature in SSO?

Make a new issue in our public SSO Github repository.

How can I keep up on new routes and features being released to ESI?

When are CREST and XML API going away?

May 8th, 2018.

Can I help with ESI development?

The UI hosted on https://esi.evetech.net is open sourced, so feel free to make that look better if you want! Here's a blog detailing how to run it locally. And as always your participation in our #esi slack channel helps inform us which directions to take ESI next.

o7

-- Team Tech Co.

Appendix

The following paths have been added or updated in ESI since the previous blog:

New routes:

  • POST /latest/universe/ids/ (v1)

Promotions from dev to latest:

  • GET /latest/alliances/{alliance_id}/ (v3)
  • GET /latest/alliances/names/ (v2)
  • GET /latest/characters/{character_id}/assets/ (v3)
  • GET /latest/characters/{character_id}/clones/ (v3)
  • GET /latest/characters/{character_id}/online/ (v2)
  • GET /latest/characters/{character_id}/roles/ (v2)
  • GET /latest/characters/{character_id}/search/ (v3)
  • GET /latest/characters/{character_id}/skills/ (v4)
  • GET /latest/characters/{character_id}/wallet/journal/ (v3)
  • GET /latest/corporations/{corporation_id}/ (v4)
  • GET /latest/corporations/{corporation_id}/assets/ (v2)
  • GET /latest/corporations/{corporation_id}/members (v3)
  • GET /latest/corporations/{corporation_id}/wallets/{division}/journal/ (v2)
  • GET /latest/corporations/names/ (v2)
  • GET /latest/search/ (v2)
  • GET /latest/characters/{character_id}/stats/ (v2)
  • POST /latest/characters/{character_id}/assets/locations/ (v2)
  • POST /latest/characters/{character_id}/cspa/ (v4)
  • POST /latest/corporations/{corporation_id}/assets/locations/ (v2)
  • DELETE /latest/characters/{character_id}/contacts/ (v2)

Get a sneak peek at what's coming to ESI by watching our deployment timeline.

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST.

]]>
Updated ESI Quick Reference For New Developers updated-esi-quick-reference-for-new-developers
http://developers.eveonline.com/blog/article/breaking-changes-to-characters-names-and-corporation-names/ Breaking Changes to /characters/names and /corporation/names 2018-06-15T17:40:08Z 2018-06-15T17:40:08Z Team Tech Co 95870245 Breaking Changes to /characters/names and /corporations/names

Over the past day it has come to Team Tech Co's attention that a few ESI routes are backed by poorly optimized software patterns. Some of these routes can and will be fixed with no disruption, but not all of them.

Now that Team Tech Co. is aware of the problem we want to rectify it swiftly. On Monday, June 18th 2018, we will be removing four routes. Two of these follow our standard promotion mechanism, whereby their legacy versions will be removed and their latest versions will be served on both latest and legacy. However, it will be necessary to remove two routes that, up until now, have been considered fully functional (albeit with their functionality fully replaced by a different route).

Of the routes you need to know about, they are:

  • All versions of /characters/names/
  • All versions of /corporations/names/
  • /v2/corporations/{corporation_id}/wallet/journal/
  • /v3/characters/{character_id}/wallet/journal/

The functionality of all versions of the /names routes is combined, extended and fully replaced by /v2/universe/names. If you are currently using the above routes you will need to migrate them to the /universe/names/ route.

The listed specific versions of the /wallet routes are already deprecated and their latest versions are not affected by the same undesirable patterns. Unless you are using the legacy versions of the listed /wallet routes you will not need to take any action. If you are unfamiliar with ESI route versioning please read this blog.

o7

-- Team Tech Co

Want to be part of the ESI conversation? Join the #esi slack channel and become a voice in our community.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

]]>
Breaking Changes to /characters/names and /corporation/names breaking-changes-to-characters-names-and-corporation-names
http://developers.eveonline.com/blog/article/a-eulogy-for-xml-crest/ A Eulogy for XML & CREST 2018-05-07T14:10:42Z 2018-05-07T14:10:42Z Team Tech Co 95870245 18 months ago we announced both ESI and our intention to retire the XML API and CREST by May 8th, 2018. This is a reminder that this retirement date is finally upon us! Tomorrow after downtime both CREST and XML API will not be usable anymore. This means any software you are using that has not moved to ESI will stop working. If you need to find an alternative to your favorite XML or CREST-only tools you can have a look at this curated list of EVE third-party software maintained by our awesome community!

The XML API was first announced in 2007 and was known as the "EVE API". The XML API was one of the first APIs to ever be released for a major popular video game and allowed the community to make amazing software to help capsuleers take the game to new heights. In 2012, a newer API was released called CREST expanding the kind of access to EVE that was available to third-party developers. As the service of these APIs comes to a close we salute them for their forward-thinking initiatives and will continue to expand on the foundation they built through the EVE Swagger Interface.

We leave you with this parting haiku from our resident poet CCP Snowedin:

one last day remains
rip to XML and CREST
long live ESI

Feel free to make your own haiku eulogies for XML API and CREST using the Twitter hashtag #LongLiveESI.

o7

-- Team Tech Co.

Want to be part of the ESI conversation? Join the #esi slack channel and become a voice in our community.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

]]>
A Eulogy For XML & CREST a-eulogy-for-xml-crest
http://developers.eveonline.com/blog/article/esi-etag-best-practices/ ESI ETag Best Practices 2018-05-02T10:28:15Z 2018-05-02T10:28:15Z Team Tech Co 95870245 o7 Space Friends,

CCP SnowedIn coming at ya with a blog detailing a recent change to ESI response headers. As some of you have already noticed (especially if you follow me on twitter, which you should), ESI now returns an ETag header in every response. This blog is to detail exactly what this means, and how you can use this in your applications to save yourself (and us) some bandwidth.

What are ETags?

Entity Tags are described in RFC7232. ESI strives to follow this RFC, and any violations should be raised as a bug on ESI-Issues.

The tl;dr of the above RFC is there are two types of ETags, strong and weak. ESI can return either. For your application the implementation is the same, you should store and use the entire ETag header value without prejudice whether or not the ETag is strong or weak.

In your application, store the ETag independently of the cache expiry. Content that has expired can be retried with the last known ETag. If the content hasn't changed, the ETag will still be the same, so you'll receive a 304 Not Modified with no response body instead of another 200 OK with data you already have.

Example use

Let's take a look at a simple example of using python's requests module to download the ESI spec. But again, just to reiterate, this concept works on every ESI route.

>>> import requests
>>> spec_url = "https://esi.tech.ccp.is/latest/swagger.json"
>>> esi_spec = requests.get(spec_url)
>>> esi_spec.status_code
200
>>> len(esi_spec.content)
921525
>>> esi_spec.headers["ETag"]
'"09f8b2541e00231360e70eb9d4d6e6504a298f9c8336277c704509a8"'
>>> etag = esi_spec.headers["ETag"]

As you can see, we just downloaded 921525 bytes (after uncompressing) to get a 200 OK response from ESI which came with a strong ETag response header. Assuming you stored this ETag somewhere (we just saved it to etag above), you could use it in the future:

>>> esi_spec_change = requests.get(spec_url, headers={"If-None-Match": etag})
>>> esi_spec_change.status_code
304
>>> len(esi_spec_change.content)
0

We know with the second request the content hasn't changed since the first time, so we can keep using the first response we received. This saves us the transfer bandwidth, uncompressing, and JSON loading this response again.

Caching considerations

Things application developers will want to consider:

  1. Store ETag values with their response data independent of the cache timers (ie; do not expire your data just because the Expires time is in the past)
  2. Request using the If-None-Match header with the last known ETag as its value
  3. When you get a 304 response, continue using the last known data. You can use the updated Expires and Date headers to determine when the next possible update will be

Browser applications

Any modern browser will automatically store and use the ETag properly. When we first implemented this feature we saw an immediate uptick in 304 responses due to this. If you are designing a fully client side web app, you don't need to worry about any of this as it will all be handled gracefully for you.

Summary

Implementing ETag based caching is a great way to prevent excessive amounts of electrons from being needlessly disturbed. Save the planet, request with the If-None-Match header today.

If you have any questions or are looking for help, jump in the #esi channel on tweetfleet slack where plenty of friendly devs are congregating and eagerly awaiting your arrival. If you're not on tweetfleet slack yet, you can get an invite here.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

]]>
ESI ETag Best Practices esi-etag-best-practices
http://developers.eveonline.com/blog/article/esi-chat-channel-deprecation/ ESI Chat Channel Deprecation 2018-04-27T15:10:11Z 2018-04-27T15:10:11Z Team Tech Co 95870245 ESI Chat Channel Deprecation

Ever since EVE's new chat backend was released on March 20th the ESI route /v1/characters/{character_id}/chat_channels/ has been returning stale data. It has been decided to officially deprecate this route, meaning that it will be labelled as "legacy" and start returning deprecation headers. This endpoint will then be removed completely on May 18th, 2018.

When will chat channel functionality be returning to ESI? At this point in time there is no ETA on the return of chat functionality in ESI or even what form it will take. All that is known is that it will look radically different than it did with /v1/characters/{character_id}/chat_channels/. You can be sure that once we know what the future of EVE chat will look like for third party developers that it will be announced here on this blog, so keep your eyes peeled!

o7

--Team Tech Co.

Want to be part of the ESI conversation? Join the #esi slack channel and become a voice in our community.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

_Migrating from the XML API? Find equivalent ESI endpoints to the XML API._

_Migrating from CREST? Find Equivalent ESI endpoints to CREST._

]]>
ESI Chat Channel Deprecation esi-chat-channel-deprecation
http://developers.eveonline.com/blog/article/esi-march-news/ ESI March News 2018-04-09T09:26:53Z 2018-04-09T09:26:53Z CCP Falcon 92532650 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018.


March has been a busy month for ESI so this blog will cover all the ESI news relevant to third party developers from the previous month.

A/B Testing

As some of you may know if you follow our twitter or participate in the #esi slack channel, ESI began proxying a percentage of requests from its current cloud provider (Google Cloud) to its new cloud provider (Amazon AWS) on March 27th.

The decision to migrate to AWS through means of A/B testing was made after Tech Co. originally dove head first into the deep end on March 15th when 100% of requests to https://esi.tech.ccp.is were redirected to https://esi.evetech.net (the URL for ESI's AWS infrastructure). Unfortunately, after about 12 hours of smooth sailing, things went downhill and we were forced to roll back to our Google Cloud infrastructure.

Following a retrospective about what went wrong, Team Tech Co. decided to take a more gentle approach and begin by only sending a small percentage of ESI requests to AWS. Thus, on March 27th, 2% of ESI requests began returning the HTTP header X-Esi-Ab-Test: 1 signifying their travels to AWS and back.

After fixing a small hiccup, ESI on AWS ingested 2% of requests with grace. On April 3rd the percentage of requests sent to AWS was bumped to 5% which is where things sit currently at the writing of this blog. The goal is to send 100% of requests to AWS by the end of April.

What does this mean for third party developers? Hopefully nothing! If things go smoothly you should not notice. However, do keep an eye out for the HTTP header X-Esi-Ab-Test: 1 in responses from ESI and please mention if it was present when you submit issues to esi-issues.

changelog.md

During ESI's lifetime, we've been keeping you up to date with future plans through our esi-issues Github project board. However, this month we hit the 50 column limit and were forced to think about how to continue letting you know about ESI updates going into the future. Instead of making a new project board, we decided it would be easier for both you and us to put all changes to ESI in a file called changelog.md at the root of our esi-issues Github repository. Enjoy!

Prioritizing esi-issues

Now that there is only one XML API -> ESI parity issue currently in progress this means that we'll be able to finally start on your never ending stream of feature requests. However, we need your help deciding what to tackle first. We would like to start a new system where thumbs up emojis are king. We are going to begin sorting feature requests by "thumbs ups", so please find the feature requests you care about and add a thumbs up to the parent comment.

ESI and chat channels

Since the release of EVE's new chat backend the ESI route /v1/characters/{character_id}/chat_channels/ ceased to return any new data. The route still functions but will only return stale data at this time. Tech Co. is working with the team behind these changes to fix this route but no ETA is known yet for when this will be fixed.

XML API Issues

Starting on March 26th the XML API started having unexpected issues and stopped being able to process a large percentage of its requests. It was brought back up on the night of March 26th and ran smoothly until March 31st, when it once again stopped being able to process any requests. After bringing it back online from this downtime it was only able to process a small percentage of requests. Since then we have put as many resources as we feel the XML API warrants but have not been able to get it recovered to its previous state.

We understand this is frustrating to those who are still transitioning from the XML API to ESI and apologize for this inconvenience. We will keep putting as many resources as we think is reasonable into fixing this problem before May 8th, but be prepared for a rocky month with the XML API.

Conclusion

There is only one month left until the official death of the XML API and CREST on May 8th. Good luck with all your XML -> ESI migrations and see you at Fanfest!

o7
-- Team Tech Co.

Appendix

The following routes have been added or updated in ESI since the previous blog:

New routes:

New versions of existing routes:

Bumped routes:

Removed routes:

  • GET /v1/corporations/{corporation_id}/assets/

Updates to existing routes:

  • system_id added to /v1/markets/{region_id}/orders/
  • asteroid_belts added to planets object in response from /v3/universe/systems/{system_id}/

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST.

]]>
ESI March News esi-march-news
http://developers.eveonline.com/blog/article/upwell-2.0-esi-structure-changes/ Upwell 2.0 ESI Structure Changes 2018-02-09T16:48:13Z 2018-02-09T16:48:13Z CCP Guard 90610935 Upwell 2.0 ESI Structure Changes

Greetings developers!

If you've been watching our dev blogs you'll know that there are some upcoming changes to structures coming on February 13th. This blog will serve as a short summary of the changes that affect ESI and how ESI is going to accommodate those changes. These structure changes will also be the catalyst for the removal of two CREST routes.

ESI changes to expect on February 13th

Before jumping into the details of what is changing in game and in ESI, it is best to give a higher level view of which ESI routes are affected and how these changes will be handled. There are two ESI routes affected by these structure changes:

Our metrics indicate that the PUT /corporations/{corporation_id}/structures/{structure_id}/ route has no usage. Therefore, we have elected not to create a replacement for the new structure vulnerability system. On release day (February 13th) this route will be labelled as legacy only, start returning deprecated headers, and will always return HTTP status code 204. It will remain this way for two weeks when it will then be completely removed and start returning 404s. If you feel strongly that a replacement is needed, please make a case for it on our esi-issues Github repository.

For more details about what is changing in game and how that will be represented in v2 of /corporation/{corporation_id}/structures, read on.

New "Full Power" and "Low Power" Modes

Structures will now be in one of two modes: "full power" or "low power". If a structure has one or more online service modules it is considered be in full power mode. If a structure is in low power mode (meaning it has no online service modules) it will affect the structure in two ways:

  1. The structure will have significantly reduced shield and armor resistances.
  2. The structure will have one less reinforcement cycle (read on for more details about reinforcement cycles).

A low power structure can be identified in the client by a label on their text bracket beside the structure name:

Low Power Client Example

The determination of whether a structure is high or lower power will be left to third party developers at this time. But there is only one question you need to answer to determine if a structure is in full power mode:

Does it have more than 0 online service modules?

If it does, it's in full power mode. If it doesn't, it's in low power mode. All services of a structure are returned in the property services from /v2/corporations/{corporation_id}/structures/ and each service has a state property saying whether is online or offline. Here is an example of a structure that is considered to be in full power mode:

{
    "structure_id": 1000000013521,
    "type_id": 35832,
    "corporation_id": 98000002,
    "system_id": 30004240,
    "profile_id": 1,
    "reinforce_weekday": 3,
    "reinforce_hour": 2,
    "state": "shield_vulnerable",
    "services": [
        {
            "name": "Manufacturing (Standard)",
            "state": "online"
        }
    ],
}

This is because the state of it's service is "online". If the service's state was "offline" it would be considered low power.

New Vulnerability and Reinforcement System

The February 13th release will see a change to the systems surrounding structure vulnerability and reinforcement. Attacks against upwell structures will now consist of a possible three phases depending on whether the structure is in full power or low power mode:

  1. An initial attack that can occur at any date and any time chosen by the attackers.
  2. An optional attack if the structure is in full power mode that will occur in the time zone of the defenders, on the day chosen by the attackers.
  3. A final battle that occurs in the time zone and day chosen by the defenders to allow them a last stand.

Under this new system, the shield hitpoints of Upwell structures will be vulnerable to attack at all times. This allows attackers to attack at any time they choose. If a structure's shields are brought to 0 it will either enter its first reinforced state if it is at full power or go straight to it's final reinforced state. Here is a flow chart detailing all the states a structure can be in with the new system:

State Flowchart

For more detailed information about the new reinforcement system please see this blog.

Each structure displayed by /v2/corporations/{corporation_id}/structures/ will have three new required properties and three optional properties to represent these in game mechanics.

The required properties are:

  • state - This string property will return which state (from the flowchart above) that a structure is in (e.g. shield_vulnerable, hull_vulnerable, etc.).

  • reinforce_weekday: This int32 property represents the day of the week when the structure will exit its final reinforcement period and become vulnerable to attack against its hull. It is represented from 0 - 6 with 0 being Monday.

  • reinforce_hour: This int32 property represents the hour of day that determines the four hour window when the structure will randomly exit its first and final reinforcement periods and become vulnerable to attack against its armor and/or hull. The structure will become vulnerable at a random time that is +/- 2 hours centered on the value of this property.

The optional properties are next_reinforce_weekday, next_reinforce_hour and next_reinforce_apply. These will only show up if a change to the reinforce_weekday and reinforce_hour have been requested in the client. All changes to reinforce_weekday and reinforce_hour take seven days to be applied in game. The date and time these changes will take effect will be the value of next_reinforce_apply.

Here is an example return from /v2/corporations/{corporation_id}/structures showing two structures where the first structure is in full power mode and has requested to change its reinforcement day and hour and the second structure is in low power mode:

[
    {
        "structure_id": 1000000013521,
        "type_id": 35832,
        "corporation_id": 98000002,
        "system_id": 30004240,
        "profile_id": 1,
        "reinforce_weekday": 3,
        "reinforce_hour": 2,
        "state": "shield_vulnerable",
        "services": [
            {
                "name": "Manufacturing (Standard)",
                "state": "online"
            }
        ],
        "next_reinforce_weekday": 1,
        "next_reinforce_hour": 4,
        "next_reinforce_apply": "2018-02-15T14:42:25Z"
    },
    {
        "structure_id": 1000000013514,
        "type_id": 35832,
        "corporation_id": 98000002,
        "system_id": 30004238,
        "profile_id": 1,
        "reinforce_weekday": 5,
        "reinforce_hour": 18,
        "state": "hull_vulnerable"
    }
]

Changes to CREST

As CREST is in maintenance only mode, it will not be updated to reflect the new system. On February 13th, both CREST endpoints will simply be deleted, and will start returning 404 errors.

  • GET /corporations/<corporationID:corporationIdType>/structures/
  • PUT /structures/<structID:integerType>/

Conclusion

We're excited to continue releasing new features in ESI along with new features in the game. Happy structure hunting and as always you can chat with us and our helpful community about these changes and more on our #esi slack channel.

o7

-- Team Tech Co


Get a sneak peek at what's coming to ESI by watching our deployment timeline.

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST.

]]>
Upwell 2.0 ESI Structure Changes upwell-2.0-esi-structure-changes
http://developers.eveonline.com/blog/article/explore-eves-data-using-the-esi-ui/ Explore EVE's Data Using the ESI UI 2018-02-08T13:34:44Z 2018-02-08T13:34:44Z Team Tech Co 95870245 Explore EVE's Data Using the ESI UI

This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018.


Quick update on ESI

Before jumping into the meat of this blog, it's important to direct your attention to a new addition to the esi-issues Github repository. With help from our community, a document detailing what constitutes a breaking change (and therefore a version bump) has been written up. In previous blogs (found here and here) we have presented how versioning is done in ESI. However, we never had a solid set of rules stating what constitutes a version bump for a given route. Now all changes that do and do not constitute a version bump have been laid out and is consulted by ESI developers when deciding whether to bump a route's version. Thank you to all community members that helped shape this document and feel free to continue making pull requests if you believe something was missed or needs to be changed!


Exploring the ESI UI

With ESI's 149 routes, finding how to get information about simple things can be a bit of an overwhelming task. This blog will step you through a basic example to show you how to use the ESI UI to do some preliminary investigation of EVE's data before committing it to code.

Finding an enemies home base

Let's say you're flying through space and get destroyed by a gentleman named CCP Zoetrope. How would you find out more info about this character using ESI so that you can return the favor? The first step is to figure out their character ID so you can look up their public info using /v4/characters/{character_id}/. To figure out CCP Zoetrope's character ID you can feed their name into /v1/universe/ids/:

ESI Get character ID from name

click here to see full sized gif

Once we have their character ID (2112625428) we can figure out some details about them using the /v4/characters/{character_id}/ route:

Get character public info

click here to see full sized gif

We see that the ID of the corporation they are in is 109299958. Using the corporation ID, we can look up what station that corporation uses as its base using /v4/corporations/{corporation_id}/ and look for the home_station_id property. Next, we will find what system that station is in (because going directly to the station may be a death sentence) using /universe/stations/{station_id}/ and looking for the system_id property:

corporation info, station info

click here to see full sized gif

Now that we have the system_id, we can plug that into /ui/autopilot/waypoint/ after authenticating to automatically set a waypoint for our client to start flying to:

set waypoint to system

click here to see full sized gif

All of this was possible to test out using only the ESI UI which allows you to know this works and continue to the next step of automating this through code.

Conclusion

ESI's UI is a powerful tool to use for investigating what's possible using ESI before touching any code. Remember that if you only have a name to work with you can use /v1/universe/ids/ and if you only have an ID you can use /v2/universe/names/. As always you can chat with us and our helpful community on our #esi slack channel

o7

--Team Tech Co.

Appendix

The following routes have been added or updated in ESI since the previous blog:

New routes:

New versions of existing routes:

Get a sneak peek at what's coming to ESI by watching our deployment timeline.

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST.

]]>
Explore EVE's Data Using the ESI UI explore-eves-data-using-the-esi-ui
http://developers.eveonline.com/blog/article/esi-quick-reference-for-new-developers/ ESI Quick Reference For New Developers 2018-01-08T18:22:08Z 2018-01-08T18:22:08Z Team Tech Co 95870245 ESI Quick Reference For New Developers

This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018.


With the recent announcement that ESI has now reached parity with XML API and CREST there will undoubtedly be newcomers to ESI (welcome!). A lot of important information about ESI is strewn about in various blogs and may be overwhelming to newcomers. This blog will act as a quick reference for the most important things every new ESI developer should know listed all in one document.

What is ESI?

A RESTful API for EVE built using the OpenAPI 2.0 specification. Keep in mind OpenAPI used to be referred to as Swagger so any previous documentation will use that term.

Will ESI have account level info?

ESI is a character oriented API and will never have account level endpoints.

Is ESI rate limited?

No, but there is an error limit. As explained here and here if you generate 100 errors in a 60 second time window your applications will start receiving the HTTP status code 420 returned for every request it makes until the 60 second time window is done. For more info on how to watch for this read the previously linked blogs.

How does authentication work with ESI?

ESI authenticated routes can be accessed through EVE's SSO (single sign on). This previous blog takes you through a step by step process for authenticated calls.

Where is the documentation for ESI?

ESI's documentation sits in its code and is automatically generated using the OpenAPI specification. You can find a list of all available routes on the ESI Swagger UI found at https://esi.tech.ccp.is/. You can expand a given endpoint and see exactly what model it returns and even try out routes directly through the UI.

Is there any documentation showing ESI equivalent endpoints to XML API and CREST?

Why yes there is!

How does versioning work in ESI?

All routes released on ESI have numbered versions (e.g. v1, v2, v3) and are tagged with one of three labels: dev, latest and legacy. The following diagram is a good top level overview of the lifecycle of a route:

lifecycle of an ESI route

The short version: always use the versioned routes in your software, never use labels. Labels can change, versions do not.

How do you know if the versioned route you are using is deprecated?

Watch for the Warning HTTP header in all your calls, you will see something similar to Warning: 299 - This endpoint is deprecated. That means next time there is a version bump on that route then the version you are using will start 404'ing.

How do you automatically generate a client for ESI using OpenAPI?

You must use the JSON version of ESI's OpenAPI specification when generating a client. You can get the full specification for all routes categorized under each of the three labels legacy, dev, and latest at https://esi.tech.ccp.is/_<label>/swagger.json where <label> should be replaced with one of the three labels used in ESI. Notice the "_" before the label, this will make it so all the routes will have the version hardcoded into the URL rather than the label.

This is easier seen than explained, so to see what the previous paragraph means first go to https://esi.tech.ccp.is/latest/swagger.json and then visit https://esi.tech.ccp.is/_latest/swagger.json. Note the differences in URLs for the routes. Always use the "undescore" route.

A previous blog post gives detailed instructions for exactly how to generate a client library. Make sure to look at this blog if you run into problems.

How can you see differences between version's specifications?

We recently released a diff UI that will show you differences per route between two different versions.

How do I report a bug or ask for a new feature in ESI?

Make a new issue in our public GitHub repository.

How do I report a bug or ask for a new feature in SSO?

Make a new issue in our public SSO Github repository.

How can I keep up on new routes and features being released to ESI?

When are CREST and XML API going away?

May 8th, 2018.

Can I help with ESI development?

The UI hosted on https://esi.tech.ccp.is is open sourced, so feel free to make that look better if you want! Here's a blog detailing how to run it locally. And as always your participation in our #esi slack channel helps inform us which directions to take ESI next.

o7

-- Team Tech Co.

Appendix

The following paths have been added or updated in ESI since the previous blog:

New routes:

  • POST /latest/universe/ids/ (v1)

Promotions from dev to latest:

  • GET /latest/alliances/{alliance_id}/ (v3)
  • GET /latest/alliances/names/ (v2)
  • GET /latest/characters/{character_id}/assets/ (v3)
  • GET /latest/characters/{character_id}/clones/ (v3)
  • GET /latest/characters/{character_id}/online/ (v2)
  • GET /latest/characters/{character_id}/roles/ (v2)
  • GET /latest/characters/{character_id}/search/ (v3)
  • GET /latest/characters/{character_id}/skills/ (v4)
  • GET /latest/characters/{character_id}/wallet/journal/ (v3)
  • GET /latest/corporations/{corporation_id}/ (v4)
  • GET /latest/corporations/{corporation_id}/assets/ (v2)
  • GET /latest/corporations/{corporation_id}/members (v3)
  • GET /latest/corporations/{corporation_id}/wallets/{division}/journal/ (v2)
  • GET /latest/corporations/names/ (v2)
  • GET /latest/search/ (v2)
  • GET /latest/characters/{character_id}/stats/ (v2)
  • POST /latest/characters/{character_id}/assets/locations/ (v2)
  • POST /latest/characters/{character_id}/cspa/ (v4)
  • POST /latest/corporations/{corporation_id}/assets/locations/ (v2)
  • DELETE /latest/characters/{character_id}/contacts/ (v2)

Get a sneak peek at what's coming to ESI by watching our deployment timeline.

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST.

]]>
ESI Quick Reference For New Developers esi-quick-reference-for-new-developers
http://developers.eveonline.com/blog/article/important-information-on-the-future-of-the-eve-api/ Important Information on the Future of the EVE API 2018-01-03T15:49:35Z 2018-01-03T15:49:35Z Team Tech Co 95870245 Greetings capsuleers,

It’s been 14 months since we first told you about the new ESI API, and today we’d like to share a little more information with you.

The ESI API is the latest result of CCP’s ongoing mission to bring you the most comprehensive API access in all of gaming. It is an API that covers all the responsibilities of both CREST and the XML API written with industry standard tooling and built to integrate with industry standard technologies. One year in, it is more comprehensive and we want to talk to you about what’s happening in the next four months.

In our first blog about ESI, released on November 8th, 2016, we stated our goal to reach parity with and shut down the legacy APIs in 18 months. We’re pleased to announce that we have reached the first milestone of this goal, and as of now ESI is at full parity!

This means that applications that are dependent on the XML API or CREST have no barriers preventing their full transition to the ESI API. Team Tech Co. will be continuing to analyze traffic and reach out to developers that are struggling to make the transition before the deadline.

The deadline in question is May 8th, 2018. On this date, we’ll be shutting down the XML and CREST APIs. Team Tech Co does not take this decision lightly, and we want you to understand why we have made this decision.

The XML API has reached a state of calcification, and expanding it further is guaranteed to break unmaintained third-party applications. Additionally, CREST and the XML API are difficult to update, and historically have lagged behind the game client in features. ESI provides us with fantastic flexibility, and enabled us to release the moon mining APIs on the same day the client feature was released.

The ESI API is built to modern industry standards and provides superior documentation compared to both its predecessors. Early on, we acknowledged that transitioning to ESI would require extra effort from the third-party community, but we were and still are certain that it would be a strategic mistake to split the API teams focus by maintaining two legacy systems. By focusing on ESI, we can maintain our current development momentum long term, and keep bringing third party developers the new features they need to make the best applications they can.

The sunsetting of the legacy APIs will impact unmaintained third-party software. It’s important that you understand that any third-party tools you wish to continue using must be migrated to ESI before this deadline. Third party software that is not migrated before this deadline will certainly break after May 8th, 2018. If you’re invested in a particular third-party software, it’s important that you reach out to the developers maintaining that software. However, it’s very likely that we have been in touch with most of the active projects in the community.

We’ve been maintaining strong communication channels since the creation of ESI. This includes active engagement with over 1800 devs in the #devfleet slack channel and over 600 in #esi. ESI has been a great community effort thus far, and we’re keen to chat with you in #esi on tweetfleet slack.

We also have an open repository for ESI bugs, features requests and design decisions at esi-issues.

Now that we’ve achieved parity, we’re enthused to start implementing all the new, fun, fancy ideas that we’ve had on the backlog since we started work on ESI.

See you in the future!

- Team Tech Co.

]]>
Important Information on the Future of the EVE API important-information-on-the-future-of-the-eve-api
http://developers.eveonline.com/blog/article/esi-endpoint-versioning-important-info-and-best-practices/ ESI Endpoint Versioning: Important Info and Best Practices 2017-12-11T10:59:45Z 2017-12-11T10:59:45Z Team Tech Co 95870245 ESI Endpoint Versioning: Important Info and Best Practices

This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


We'd like to start this blog with an important announcement that will affect a large swath of EVE third party developers

On December 20th, 2017, the following dev routes will be promoted to latest:

  • GET /v3/alliances/{alliance_id}/
  • GET /v2/alliances/names/
  • GET /v3/characters/{character_id}/assets/
  • GET /v3/characters/{character_id}/clones/
  • GET /v2/characters/{character_id}/online/
  • GET /v2/characters/{character_id}/roles/
  • GET /v3/characters/{character_id}/search/
  • GET /v4/characters/{character_id}/skills/
  • GET /v3/characters/{character_id}/wallet/journal/
  • POST /v2/characters/{character_id}/assets/locations/
  • POST /v4/characters/{character_id}/cspa/
  • DELETE /v2/characters/{character_id}/contacts/
  • GET /v4/corporations/{corporation_id}/
  • GET /v2/corporations/{corporation_id}/assets/
  • GET /v3/corporations/{corporation_id}/members/
  • GET /v2/corporations/{corporation_id}/wallets/{division}/journal/
  • GET /v2/corporations/names/
  • POST /v2/corporations/{corporation_id}/assets/locations/
  • GET /v2/search/

This means any software not using the numbered versions of these routes will suffer from backwards incompatible changes on December 20th. To avoid disruption, we offer the following checklist:

  1. Check to make sure your software is only calling versioned ESI routes. (e.g v1, not latest, dev or legacy).

  2. Check to make sure that your code is not using any versioned routes labelled only as legacy from the above list.

  3. Optionally, upgrade to the versioned routes above (currently labelled as dev) and make the necessary changes to your code. This is highly suggested.

Read on for more in depth instructions.

How ESI routes are versioned

Before digging deeper into the above checklist it is best to do a recap on the lifecycle of a route. The following is a diagram showing how a route is versioned and labelled from its inception:

lifecycle of a route

This diagram should show you why it is bad to use the legacy, latest or dev labels directly in your ESI URLs and why you should always use versioned routes. If you're using labels in your URLs the spec underneath those labels changes dramatically during route promotion.

December 20th Checklist

1. Check to make sure your software is only calling versioned ESI routes.

Not only should you do this for the above 19 routes, but it's best if you use the versioned alternative for all ESI routes inside your software. This is as simple as doing a find and replace on the ESI URLs you are calling.

Good: https://esi.tech.ccp.is/v2/alliances/{alliance_id}/
Not good: https://esi.tech.ccp.is/latest/alliances/{alliance_id}/

2. Check to make sure that your code is not using any versioned routes labelled only as legacy.

Here is a list of versions of the above 19 routes that will start 404'ing on December 20th:

  • GET /v1/alliances/{alliance_id}/
  • GET /v1/characters/{character_id}/assets/
  • GET /v1/characters/{character_id}/search/
  • GET /v2/characters/{character_id}/skills/
  • GET /v1/characters/{character_id}/wallet/journal/
  • GET /v2/corporations/{corporation_id}/

If you are currently using any of these versioned routes (thanks for using versions!) in your code then you will have to update your code to support a current version before December 20th.

If you want to know if any of the other versioned routes you are depending on are considered deprecated there are two ways to do that:

  1. Look at https://esi.tech.ccp.is/ui/?version=legacy. Any routes that have a strike through their paths will be deleted next time a new version is promoted to latest.

  2. Watch for a Warning HTTP header. This header informs you that the route you are using has been deprecated.

3. Optionally, upgrade the above 19 routes to their dev versions

If you update your software to use all the versioned routes listed above then your software will be even more future proofed! However, bumping up to a new version means that the spec has changed and you will have to modify your code to deal with that. Luckily, Team Tech Co. just released a new UI to help you see a diff between all routes under specific labels or versions: https://esi.tech.ccp.is/diff/latest/dev/.

If a route has a new version available in dev it will also now show in the ESI UI like this:

New route screenshot

Clicking that hyperlink will take you to the new diff UI for the given route.

Why mass promote all these 19 routes to latest?

As was shown in the route lifecycle diagram, the time that a versioned route is supposed to be labelled with dev should be six weeks before it's promoted to latest. Why are there 19 routes that have not been promoted to latest then? The simple answer is that during the race to parity we forgot to promote some endpoints along the way. The mass promotion on December 20th is our way to correct this oversight and restart the proper promotion workflow. We are actively developing internal tools to help us keep on top of this in the future.

A reminder for those using swagger codegen

If you are automatically generating clients using Swagger codegen, remember to ALWAYS use the "underscore" ESI swagger.json route when choosing a URL to generate a client from. As an example, instead of using https://esi.tech.ccp.is/latest/swagger.json, use https://esi.tech.ccp.is/_latest/swagger.json. Notice the _ placed before "latest" in the latter URL. This will make your auto generated client use the versioned routes instead of labels in its ESI URLs. Remember you can also generate a client using the versioned routes from both dev and legacy by using _dev or _legacy like _latest was used in the example.

Conclusion

In conclusion:

  • On December 20th there will be a mass promotion of all versioned routes currently in dev (approximately 19 of them) to latest.
  • This mass promotion will affect you if any of your code is using latest in the URL for any of those 19 routes.
  • Follow the checklist given to ensure a smooth transition.

As always, you can discuss all things ESI on the tweetfleet slack (use this link to join) in the #esi channel.

o7

-- Team Tech Co.

Appendix

The following paths have been added or updated in ESI since the previous blog:

Get a sneak peek at what's coming to ESI by watching our deployment timeline

Migrating from the XML API? Find equivalent ESI routes to the XML API.

Migrating from CREST? Find Equivalent ESI routes to CREST

]]>
ESI Endpoint Versioning: Important Info and Best Practices esi-endpoint-versioning-important-info-and-best-practices
http://developers.eveonline.com/blog/article/simplify-esi-queries-by-using-search/ Simplify ESI Queries By Using /search/ 2017-11-08T00:00:00Z 2017-11-08T00:00:00Z Team Tech Co 95870245 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


How would you find the type ID of a Rifter using ESI?

It is easy for a new ESI developer to feel overwhelmed by this simple question. How should it be solved? It's likely that the solution an ESI novice would come up with would be to:

However, that would be ridiculous! ESI supplies an endpoint for just such a case: /search/.

What is /search/?

/search/ is a public endpoint that allows you to enter a search string and will return a list of IDs containing the given string in a given category. The following categories can be selected and are passed in as a query parameter (categories that require further explanation are followed by a "-"):

  • agent - Used to find NPC characters that give out missions.
  • alliance
  • character
  • constellation
  • corporation
  • faction - Used to find NPC factions.
  • inventory_type - Used to find the type ID of any entity in the game that could be considered an "item" (e.g. ships, blueprints, modules, etc.).
  • region
  • solar_system
  • station - This doesn't cover POS's or citadels.

There is also an authenticated search endpoint: /characters/{character_id}/search/. This authenticated endpoint adds one more category named structure and will allow you to search for upwell structures your character has access to.

Now let's look at an example of using /search/ to answer the question posed at the beginning of this blog.

Getting a Rifter's type ID using /search/

Which of the search categories would be best to use when searching for a Rifter's type ID? Looking at the list of categories it's apparent that we should search using the inventory_type category. Let's make a request to /search/ by using curl with the query parameters categories=inventory_type and search=Rifter.

$ curl "https://esi.tech.ccp.is/v2/search/?categories=inventory_type&search=Rifter"

This request returns:

{
    "inventory_type":[
        587,
        691,
        33664,
        33666,
        34748,
        34749,
        35493,
        35494,
        35495,
        35496,
        35497,
        35498,
        35499,
        35500,
        36411,
        36754,
        36801,
        36992,
        36993,
        40470,
        44166,
        45554,
        45936
    ]
}

All the IDs returned have the word "Rifter" in them and we can check what they are by dropping them into /universe/types/{type_id}/. Let's take a look
at type ID 587 using curl:

$ curl "https://esi.tech.ccp.is/v3/universe/types/587/"

which returns:

{
  "type_id": 587,
  "name": "Rifter",
  "description": "The Rifter is a very powerful combat frigate and can easily tackle the best frigates out there. It has gone through many radical design phases since its inauguration during the Minmatar Rebellion. The Rifter has a wide variety of offensive capabilities, making it an unpredictable and deadly adversary.",
  "published": true,
  "group_id": 25,
  "market_group_id": 64,
  "radius": 31,
  "volume": 27289,
  "packaged_volume": 2500,
  "capacity": 140,
  "portion_size": 1,
  "mass": 1067000,
  "graphic_id": 46,
  "dogma_attributes": [
    ... // Denotes that there is content but it is being reduced for example's sake.
  ]
  "dogma_effects": [
    ...
  ]
}

This is exactly what we were looking for! So what are the other type IDs? 681 is a "Rifter Blueprint", 33664 is a "Rifter Nefantar Edition Blueprint" and the rest are left up to the reader to look into if they feel so compelled. As you can see, any type ID that has the word "Rifter" is shown. How would we restrict this search to only return type ID 587? By passing in strict=true as a query parameter! The strict query parameter will look for exact matches only, let's try it:

$ curl "https://esi.tech.ccp.is/v2/search/?categories=inventory_type&search=Rifter&strict=true"

which returns:

{
  "inventory_type": [
    587
  ]
}

Using multiple categories at a time

It is possible to pass up to 10 categories at once by using a comma (",") separated list as the value of the categories query parameter. Let's do a search for the word "Awesome" in all categories:

$ curl "https://esi.tech.ccp.is/v2/search/?search=Awesome&categories=agent,alliance,character,constellation,corporation,faction,inventory_type,region,solar_system,station"

This returns (edited for length):

{
    "character":[
        ...452 character IDs...
    ],
    "corporation":[
        ...47 corporation IDs...
    ]
    "station":[
        61000138
    ]
}

There are 452 characters, 47 corporations and 1 station that contain the word "Awesome" (as of the writing of this blog). You can get more info on the characters returned by dropping their IDs into the /characters/{character_id}/ endpoint. You can drop the corporation IDs into /corporation/{corporation_id/} and you can drop the station ID into /universe/stations/{station_id}/.

Conclusion

The /search/ endpoint can simplify your queries and will save you the pain of iterating through thousands of IDs until you find the one you want. Go forth and search!

As always, you can discuss all things ESI on the tweetfleet slack (use this link to join) in the #esi channel.

o7

-- Team Tech Co.

Appendix

The following paths have been added or updated in ESI since the previous blog:

Get a sneak peek at what's coming to ESI by watching our deployment timeline

Migrating from the XML API? Find equivalent ESI endpoints to the XML API.

Migrating from CREST? Find Equivalent ESI endpoints to CREST

]]>
Simplify ESI Queries By Using /search/ simplify-esi-queries-by-using-search
http://developers.eveonline.com/blog/article/developer-license-agreement-update-2017-10/ Updates to the Developer License Agreement - 2017-10 2017-10-19T14:50:00Z 2017-10-19T14:50:00Z Team Tech Co 95870245 Hello developers

CCP is making a number of changes to the developer agreement. The primary purpose of these changes is to allow for persons other than the original developer to enter into the agreement and get access to the resources for EVE Online made available by CCP. An example of such a user might be someone that has downloaded an application written by someone else. As the author of the application can't include a client ID and secret in the application (as they become public), the user will need to provide their own. Since the end user is not the original author of the application, they were not allowed to sign the agreement.

The full updated Developer License Agreement can be found here. If you just want to see the changes, they're detailed below. Additions are in bold, removed text is marked with a strikethrough

BACKGROUND STATEMENT
CCP is the owner of the massively multiplayer online game EVE Online® ("EVE"). As a "Developer", you desire to make, deploy and/or administer one or more software applications to be used in connection with EVE.

This is a change to maintain continuity with the change to the definition of Application below

1.2
"Application" means any application software designed, deployed or administered by Developer to provide functionality for or in conjunction with EVE via CRESTESI or any other CCP tool, whether the software is installed on an individual computer (e.g., a desktop or laptop), a mobile device (e.g., a smart phone or tablet), or offered as a service remotely (e.g., accessed via a web site or direct connection).

This is a fairly significant change. Previously you were only able to enter into the Developer License Agreement if you were the original author of the application you were using with CCP Tools. With this change, anyone that uses an application is now able to sign the agreement. This includes people both running applications on their own computers, as well as people who are deploying their own instances of applications for use by their corporation, alliance or other group.

We have also updated references to CREST to ESI throughout the license agreement, as ESI will be the primary means of accessing information about the state of the game going forward.

Section 1.6 (the definition of CREST) has been removed. This was not needed anymore with the removal of references to CREST, and a new definition for ESI has been added as section 1.8.

1.7
"Derivative Works" means any artistic or literary creation derived from EVE, the CCP Marks, the Licensed Materials, or any part thereof. Notwithstanding the foregoing, CCP acknowledges that it does not own any each Application is owned by the Developer, and no Application shall not be considered a derivative work within the meaning of this paragraph.

Previously this stated that CCP acknowledged that the application was owned by the Developer and would not be considered a derivative work. As we have changed the agreement to now allow persons other than the original developer to sign the agreement, this has been changed to CCP not making any claims about the ownership of the application.

2.8
Subject to CCP's compliance with the terms and conditions of this Agreement, to the extent Developer designed the Application, Developer grants CCP a limited, non-exclusive, worldwide, non-transferable license and right during the Term to use and display the Application and view any publicly available materials associated therewith (e.g., documentation). CCP's rights granted under this Section shall be limited to actions that: (1) help ensure the technical effectiveness and integration of the Application with EVE or the Licensed Materials, (b) help troubleshoot the Application, EVE, or the Licensed Materials, (3) help ensure Developer's ongoing compliance with this Agreement, or (4) exercise CCP's right to market to the Application in accordance with Section 3.

This section has been changed to ensure that someone using an application created by someone else isn’t required to give over license they aren’t able to grant.

7.6
CCP recognizes that it is not the owner Developer is the exclusive of the Applications(s), and CCP will not contest Developer’s ownership of the Application(s). In addition, CCP will not register or attempt to register any intellectual property right or other ownership right in the Application(s) (except to the extent necessary to protect CCP's ownership of EVE, the CCP Marks, and the Licensed Materials).

Just like section 1.7, this change is due the agreement allowing someone other than the original designer of the application to sign the agreement, while keeping the intention of CCP not making any claims to own the application

9.3
Developer represents and warrants that each Application is Developer’s own original work, or an original work of the organization of which Developer is an authorized representative, and that such (a) has been created by a party who has entered into the Developer License Agreement with CCP and (b) does not violate the copyright or other any intellectual property rights of any third party.

This is the main change. There is no longer a requirement that the application must be the original work of the developer, meaning that other people can now sign the agreement to get access to things like the SSO or ESI endpoints requiring authorization. However, to ensure that there are no edge cases where someone that didn’t sign the agreement wrote the application, we require that any applications used were created by someone that is party to the agreement. For this reason, we now recommend that application creators add a line somewhere in their application or their website stating that they have signed the agreement, so that users have confidence that they can use your application. An example line that would work would be "This application has been created under the EVE Developer License Agreement."

This isn’t really a new restriction, as previously you could only sign the agreement if you were the original author of the application, which would mean you would have the authority to assign rights and licenses to CCP as required, as well as ensure your application conformed to any requirements laid out in the agreement.

In addition to these changes, there are two sections (10.3 and 5.5) where we have corrected incorrectly numbered sections. There are no changes to the text of these sections or the meaning of them.

With these changes, we hope to see more people using both the SSO and ESI with confidence that they won't have any issues from CCP. If you have any further questions, please come find us in the #devfleet channel on the Tweetfleet Slack.

]]>
Developer License Agreement Update developer-license-agreement-update-2017-10
http://developers.eveonline.com/blog/article/esi-mining-ledger/ Mining Ledgers in ESI 2017-10-08T00:00:00Z 2017-10-08T00:00:00Z Team Tech Co 95870245 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


Moon mining is coming to EVE Online on October 24th in the Lifeblood expansion and, for the first time ever, new ESI endpoints will accompany this new feature's release! The new endpoints will be onlined a few hours after downtime on October 24th for Tranquility. We have already released revamped moon mining onto our test server, Singularity, along with the mining ledger ESI endpoints so that you can begin making software for this feature now.

There are three new endpoints to accompany mining ledgers. Let's look a bit closer at each endpoint and how to use them.

Personal Mining Ledgers

There is one endpoint in ESI connected to personal mining ledgers:

GET /v1/characters/{character_id}/mining/

Cache time: 600 seconds (10 minutes)

Required scope: esi-industry.read_character_mining.v1

Pagination: 1000 results per page, returns X-pages HTTP header

With the addition of revamped moon mining in EVE, every mineral you mine will now be recorded into mining ledgers. This endpoint allows you to look at your character's personal mining ledger history for the past 90 days.

Example

First get an access token with the esi-industry.read_character_mining.v1 scope. Make sure to use the SSO base URL https://sisilogin.testeveonline.com if you're doing this against Singularity.

Then, make the following call and replace {character_id} with your character ID and {access_token} with the access token you received:

$ curl --header "Accept: application/json" --header "Authorization: Bearer {access_token}" https://esi.tech.ccp.is/v1/characters/{character_id}/mining/?datasource=singularity

Note: the datasource parameter in this example is only needed when getting data from Singularity before the Lifeblood expansion is released.

Here's a look at an example response for this endpoint:

[
    {
        "date": "2017-09-21",
        "solar_system_id": 30002537,
        "type_id": 1227,
        "quantity": 5092
    },
    {
        "date": "2017-09-22",
        "solar_system_id": 30002537,
        "type_id": 1227,
        "quantity": 408
    },
    {
        "date": "2017-09-23",
        "solar_system_id": 30002537,
        "type_id": 1227,
        "quantity": 1872
    },
    {
        "date": "2017-09-24",
        "solar_system_id": 30002537,
        "type_id": 1227,
        "quantity": 432
    },
    {
        "date": "2017-09-21",
        "solar_system_id": 30002537,
        "type_id": 1228,
        "quantity": 1000
    },
    {
        "date": "2017-09-21",
        "solar_system_id": 30002538,
        "type_id": 1227,
        "quantity": 1000
    }
]

As you can see, mining ledger quantities are aggregated per unique combination of date, solar_system_id and type_id.

You can get more info on the given solar_system_id by dropping it into the /universe/system/{system_id}/ endpoint. You can also drop the given type_id into the /universe/types/{type_id}/ endpoint to see more details on the mineral that was mined.

Corporation Mining Ledgers

The corporation mining ledger endpoints in ESI exist to allow a corporation to track the mining of asteroids that are being observed by a corporation's Upwell Refineries. They do not give the ability to monitor what each member in a corporation is mining. To better understand this "observer" concept, let's look at how moon mining will work on a corporation level in-game:

  • A corporation deploys a Upwell Refinery close to a moon.
  • The refinery is then fitted with a special moon drilling service module (only one refinery per moon can be fit with a moon drilling module).
  • The moon drilling module blasts a chunk of the moon away from its surface and drags it towards the structure. This process takes between one and several weeks.
  • Once the chunk of the moon is close enough to the refinery it will be blasted into a small asteroid field which can then be mined.
  • The refinery itself will observe who mines this asteroid field and update its mining ledger.

If you would like a deeper understanding of Upwell Refineries you can read this blog.

ESI exposes two endpoints that are connected to corporation mining ledgers:

GET /corporation/{corporation_id}/mining/observers/

Cache time: 3600 seconds (1 hour)

Required scope: esi-industry.read_corporation_mining.v1

Required role: Accountant

Pagination: 1000 results per page, returns X-pages HTTP header

This endpoint exists to list all of your corporation's structures capable of observing moon mining and keeping ledgers. In this case, the list of observers will be a list of Upwell Refineries that a corporation owns which have mining events. Refineries with no mining events will not be shown on this list.

Example

First get an access token with the esi-industry.read_corporation_mining.v1 scope. Make sure to use the SSO base URL https://sisilogin.testeveonline.com if you're doing this against Singularity.

Then, make the following call and replace {corporation_id} with your corporation ID and {access_token} with the access token you received:

$ curl --header "Accept: application/json" --header "Authorization: Bearer {access_token}" https://esi.tech.ccp.is/v1/corporation/{corporation_id}/mining/observers/?datasource=singularity

Note: the datasource parameter in this example is only needed when getting data from Singularity before the Lifeblood expansion is released.

Here's a look at the response for the in-game CCP Corporation on Singularity:

[
    {
        "last_updated": "2017-09-21",
        "observer_id": 1025242944478,
        "observer_type": "structure"
    },
    {
        "last_updated": "2017-10-03",
        "observer_id": 1025246025947,
        "observer_type": "structure"
    },
    {
        "last_updated": "2017-09-28",
        "observer_id": 1025246389128,
        "observer_type": "structure"
    },
    {
        "last_updated": "2017-09-30",
        "observer_id": 1025249693147,
        "observer_type": "structure"
    }
]

Each observer_id is the ID for a Upwell Refinery that has logged mining events. The CCP corporation owns more than four Upwell Refineries on Singularity, but these do not show up because they have not been participating in any mining activity. You will use the observer_id in the endpoint described below. To get more info on the given structure behind the observer_id you can drop it into the /universe/structures/{structure_id}/ endpoint.

GET /corporation/{corporation_id}/mining/observers/{observer_id}/

Cache time: 3600 seconds (1 hour)

Required scope: esi-industry.read_corporation_mining.v1

Required role: Accountant

Pagination: 1000 results per page, returns X-pages HTTP header

Given an observer_id from the above endpoint, this will list all mining events recorded in the mining ledger for the given observer. Using this, you can tell who mined which type of mineral and which corporation that character belonged to at the time of mining. This endpoint is purposely cached for an hour so that you cannot get real-time updates on whether someone is ninja mining your corporation's minerals. You'll have to police that in-game!

Example

First get an access token with the esi-industry.read_corporation_mining.v1 scope. Make sure to use the SSO base URL https://sisilogin.testeveonline.com if you're doing this against Singularity.

Then, make the following call and replace {corporation_id} with your corporation ID, {observer_id} with an ID of an Upwell Refinery and {access_token} with the access token you received:

$ curl --header "Accept: application/json" --header "Authorization: Bearer {access_token}" https://esi.tech.ccp.is/v1/corporation/{corporation_id}/mining/observers/{observer_id}/?datasource=singularity

Note: the datasource parameter in this example is only needed when getting data from Singularity before the Lifeblood expansion is released.

Here's a look at the response for an Upwell Refinery that the in-game CCP Corporation has on Singularity:

[
    {
        "last_updated": "2017-09-21",
        "character_id": 645704275,
        "recorded_corporation_id": 109299958,
        "type_id": 45491,
        "quantity": 5130
    },
    {
        "last_updated": "2017-09-21",
        "character_id": 95633128,
        "recorded_corporation_id": 98407839,
        "type_id": 45491,
        "quantity": 11832
    },
    {
        "last_updated": "2017-09-21",
        "character_id": 95633128,
        "recorded_corporation_id": 98407839,
        "type_id": 45492,
        "quantity": 11832
    },
    {
        "last_updated": "2017-09-21",
        "character_id": 95633128,
        "recorded_corporation_id": 98407839,
        "type_id": 1231,
        "quantity": 3926
    }
]

As we can see, each entry in the mining ledger has a character_id of the character that did the mining, a recorded_corporation_id showing which corporation that character was part of when they did the mining, a type_id for the mineral type that was mined and quantity to show how much of that mineral was mined. This endpoint shows mining events going back 30 days.

You can drop the character_id into the /characters/{character_id}/ endpoint for more details about a given character ID. recorded_corporation_id can be dropped into the /corporations/{corporation_id}/ endpoint and type_id can be dropped into the /universe/types/{type_id}/ endpoint.

What happens when an Upwell Refinery changes ownership?

If corporation A sells its Upwell Refinery to corporation B then corporation A will still see this structure listed in the response from the /corporation/{corporation_id}/mining/observers/ endpoint. However, if corporation A looks up mining events for the sold structure they will only see events that happened in the past 30 days while it was under corporation A's ownership. Any mining events that happen after the transition to corporation B will only be viewable by corporation B.

Conclusion

Team Tech Co. is excited to release new endpoints along with a new feature and we hope this overview can give you a deeper understanding of how to use these three mining ledger endpoints. We look forward to seeing what you will do! As always, you can discuss these endpoints in the tweetfleet slack (use this link to join) in the #esi channel.

o7

-- Team Tech Co.


Nice and ESI (Easy) Task.async_stream -- A blog about concurrently fetching all of EVE's solar systems via ESI using Elixir by CCP AquarHEAD.

Appendix

The following paths have been added or updated in ESI since the previous blog:

  • GET /latest/corporations/{corporation_id}/wallets/{division}/transactions/ (v1)
  • GET /latest/corporations/{corporation_id}/assets/ (v1)
  • GET /latest/corporations/{corporation_id}/blueprints/ (v1)
  • GET /latest/corporations/{corporation_id}/titles/ (v1)
  • GET /latest/characters/{character_id}/assets/ (v1, spec breaking) -- Migrated to a pagination style
  • GET /latest/characters/{character_id}/calendar/{event_id}/attendees/ (v1)
  • GET /latest/characters/{character_id}/wallet/journal/ (v2)
  • GET /latest/characters/{character_id}/blueprints/ (v2)
  • GET /latest/universe/types/{type_id}/ (v3)
  • POST /latest/characters/{character_id}/assets/names/ (v1)
  • POST /latest/characters/{character_id}/assets/locations/ (v1)

Get a sneak peek at what's coming to ESI by watching our deployment timeline

]]>
ESI mining ledger esi-mining-ledger
http://developers.eveonline.com/blog/article/esi-universe-names-endpoint-update/ ESI Universe Names ID Range Update 2017-10-02T14:52:34Z 2017-10-02T14:52:34Z Team Tech Co 95870245 Greetings developer capsuleers,

Team Tech Co. previously blogged about a set of range checks that we placed on /universe/names/ and /characters/affiliation/ that made these ESI endpoints return blanket 404s for the range of IDs encompassing Dust characters and legacy EVE characters and corporations. As of today, with the enabling of error limiting, we have now removed these range checks.

This means that any 404s you get from /universe/names/ and /characters/affiliation/ will be legitimate 404s. Remember that given a set of IDs if any of those IDs are not valid you will get a 404 even if some of the other IDs in the set are valid.

As always, you can come talk to us in the tweetfleet slack (use this link to join) in the #esi channel if you have further questions.

-Team Tech Co.

]]>
ESI Universe Names Endpoint Update esi-universe-names-endpoint-update
http://developers.eveonline.com/blog/article/esi-error-limits-go-live/ ESI Error Rate Limiting Goes Live On Monday 2017-09-27T16:34:40Z 2017-09-27T16:34:40Z Team Tech Co 95870245 Hello space developers,

As stated previously, error limiting in ESI has been imminent since August 29th. Starting after downtime on Monday, October 2nd, Team Tech Co. will officially turn on the ESI error limit functionality.

If you're worried about invoking the wrath of the error limiter, you still have the weekend to check your application and clean up your code (you are backing off when you get errors, right?). The way to know if your software will be error limited after Monday is to look for the HTTP header X-Esi-Error-Limited in responses coming from ESI.

There are two HTTP headers being returned from ESI that will help ensure your app never hits the error limit. These are X-Esi-Error-Limit-Remain and X-Esi-Error-Limit-Reset. X-Esi-Error-Limit-Remain will let you know how many more errors you can make within the window of time defined by X-Esi-Error-Limit-Reset, which indicates the number of seconds until the end of the current error window.

What counts as an error? Any response in the 4xx or 5xx range will count towards your error limit.

If you have any questions or have a use case where this error limit would legitimately restrict your software then come talk to us on the #esi channel in the tweetfleet slack!

  • Team Tech Co.

]]>
ESI Error Limits Go live esi-error-limits-go-live
http://developers.eveonline.com/blog/article/esi-concurrent-programming-and-pagination/ ESI, Concurrent Programming, and Pagination 2017-09-08T14:42:00Z 2017-09-08T14:42:00Z CCP Zoetrope 2112625428 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


This blog will cover the concept of concurrent programming and how to use this model of programming to improve your software's performance while pulling data from ESI. If you are already familiar with the concept of concurrent programming here is a ESI pagination TLDR: Use the X-Pages header returned from paginated ESI endpoints to determine how many calls are needed to get all data from a given endpoint.

Prerequisites

This blog includes examples of Linux commands as well as Python code, therefore if a reader wants to follow this blog exactly then they will have to do so on a Linux based system and use Python 3. It is also assumed that the user has both pip and virtualenv installed.


Have you ever used ESI's /markets/{region_id}/orders/ endpoint? Did you know that this endpoint is paginated and allows you to retrieve more than just the first 10,000 orders? Did you know that there are more ESI endpoints than this that support pagination? If the answer to one or more of these questions was no, then this blog is for you! If you answered yes but you have never been quite sure how you are supposed to utilize pagination in ESI, then this blog is also for you! Using the /markets/{region_id}/orders/ endpoint this blog will walk you through getting all orders from a specific region in a concurrent manner. This method can then be used for other paginated ESI endpoints or applied generally to concurrently fetch information from multiple endpoints.

How ESI Handles Pagination

ESI handles pagination the same for most[1] endpoints, here's a high-level example of getting the paginated data in the market orders endpoint:

  1. Send a request to https://esi.tech.ccp.is/v1/markets/10000042/orders/ to get the first 10,000 orders in the Metropolis region.

  2. ESI returns a response that contains 10,000 orders along with the header X-Pages which has a value of 10. This is telling you that all active orders in this region are split into 10 pages.

  3. Knowing that there are ten pages, you can now send nine more requests (because you already have page 1) starting at page 2 and incrementing up to 10. The way you do this is by adding a page query parameter to the URL.

  4. After making these ten requests you should have all orders in Metropolis.

Remember that the number returned by X-Pages will vary depending on the time of day and region, and is simply a way of letting you know how much more data there is. Each paginated endpoint in ESI will possibly have a different max number of items per page which is indicated by the maxItems property in a given endpoint's swagger spec.

How to tell if an ESI endpoint is paginated

If you want to know if a particular ESI endpoint is paginated you will see "page" listed as a parameter for a given endpoint in its swagger spec.

Setting up your Environment

Before moving on you must set up an appropriate environment. In a directory of your choice, make a new virtual environment:

$ virtualenv .venv --python=python3

Activate this virtual environment:

$ source .venv/bin/activate

Next, run the following pip commands to install the libraries necessary to run the examples in this blog:

(.venv)$ pip install grequests

Getting Market Orders Synchronously using Python

We're going to first get all market orders in Metropolis (region ID 10000042) in a sequential manner. The following is pseudocode to show what we will be doing:

call https://esi.tech.ccp.is/v1/markets/10000042/orders/
extract the X-Pages header from the return
for page_number in pages - 1
    call https://esi.tech.ccp.is/v1/markets/10000042/orders/?page=page_number+1
combine the data from all calls

If the value returned by X-Pages was 3, then we could visualize this program like so:

Synchronous market orders diagram

where each colored block represents a task over time.

Make a new file called get_sequential.py and paste this code inside:

Make sure you're still inside the virtual environment you created earlier and run this by doing

(.venv)$ python get_sequential.py

This code will display how long it took to complete all the requests needed to get all of the orders and how many orders it retrieved. Here is the output from running it on my machine:

Elapsed time for 9 requests was: 4.024033069610596
Got 91,064 orders.

As we can see it took about 4 seconds to get 9 pages sequentially (remember that it skips calling the first page again). Why is that? It's because, for every request sent, the requests library waits for the response from the endpoint. Basically, a lot of the time spent is because of network latency. This is easier to visualize by adding wait time to the previous visualization (the gray areas signify wait time):

Synchronous market orders diagram with wait time

In this case, "wait time" is considered to be the time that our code is just waiting for a response from ESI. We could instead take advantage of this wait time and let our code perform more tasks that do not depend on the data returned from ESI. If we were to do this, the code would be considered concurrent.

Getting Market Orders Concurrently using Python

How would retrieving paginated market orders work in a concurrent manner? Which tasks could be done without needing to know about the other? Because the wait time is mostly network related, we could instead start each ESI call one after the other and consume their returns at a later time. Here's pseudocode to show how we will approach this concurrently (comments are preceded by the # symbol):

call https://esi.tech.ccp.is/v1/markets/10000042/orders/
wait for the response from ESI # because we need the value of X-Pages
extract the X-Pages header from the return
for page_number in pages - 1
    call https://esi.tech.ccp.is/v1/markets/10000042/orders/?page=page_number+1
    defer waiting for the response
consume all responses returned from ESI
combine all the orders

Here is a modified version of the previous visualization to represent this:

Concurrent market order diagram

Remember that this is a simplified visualization and may not represent exactly how these given tasks are scheduled by gevent. Read more about gevent and greenlets here.

How is this done using Python? For this blog we will be using a library called grequests that in turn depends on the libraries gevent and requests. gevent is a networking library for Python that allows network calls to be started, suspended, and resumed independently in an event loop. grequests combines the requests library with gevent, essentially allowing HTTP requests to be started and consumed at different times. Put the following in a file called get_concurrent.py:

As can be seen, grequests does all the heavy lifting as far as the concurrent logic is concerned, and the magic of the concurrent calls is done particularly in the grequests.map() method that is called inside the concurrent_requests method. grequests.map() simply needs Request objects so that it can then call all endpoints and consume the returns from ESI at a later time. If you're interested in understanding further how grequests.map works you can read the code.

Run this by doing the following command:

(.venv)$ python get_concurrent.py

and you should get output similar to:

Elapsed time for 9 requests was: 0.554999828338623
Got 91,064 orders.

As we can see, the time it took to make the requests dramatically dropped. When this was done sequentially it took about 4 seconds and when done concurrently it took about 0.5 seconds.

Conclusion

Making calls to ESI in a concurrent way will dramatically speed up your software. Hopefully, the demonstration in this blog can serve as a jumping off point for you to continue on a concurrent path. There are currently only two other endpoints in ESI that handle pagination with the X-Pages header. The first is the /character/{character_id}/blueprints/ endpoint which recently got the addition of pagination and is currently only in the /v2 and /dev namespace. The second endpoint is /markets/{region_id}/types.

There are other paginated endpoints in ESI but they do not support the X-Pages header as of yet. However, Team Tech Co. plans to expand these endpoint's functionality to use the X-Pages header. These endpoints are:

o7

  • CCP Zoetrope

[1] The following endpoints could be considered paginated but operate differently:


Appendix

The following paths have been added or updated in ESI since the previous blog:

  • GET /latest/corporations/{corporation_id}/wallets/ (v1)
  • GET /latest/corporations/{corporation_id}/membertracking/ (v1)
  • GET /dev/alliances/{alliance_id}/ (v3)
  • GET /latest/fw/systems/ (v1)
  • GET /latest/fw/leaderboards/corporations/ (v1)
  • GET /latest/fw/leaderboards/characters/ (v1)
  • GET /latest/fw/leaderboards/ (v1)
  • GET /latest/characters/{character_id}/planets/{planet_id}/ (v3)
  • GET /latest/characters/{character_id}/notifications/contacts/ (v1)
  • GET /latest/universe/systems/{system_id}/ (v3)
  • GET /dev/characters/{character_id}/blueprints/ (v2)
  • GET /dev/characters/{character_id}/roles/ (v2)
  • GET /latest/corporations/{corporation_id}/killmails/recent (v1)
  • GET /latest/corporations/{corporation_id}/wallets/{division}/journal (v1)
  • GET /latest/corporations/{corporation_id}/divisions/ (v1)
  • GET /latest/corporations/{corporation_id}/members/limit (v1)
  • GET /latest/corporations/{corporation_id}/contacts (v1)

Get a sneak peek at what's coming to ESI by watching this board.

]]>
ESI, Concurrent Programming, and Pagination esi-concurrent-programming-and-pagination
http://developers.eveonline.com/blog/article/error-limiting-imminent/ Error Rate Limiting Imminent 2017-08-29T18:06:04Z 2017-08-29T18:06:04Z Team Tech Co 95870245 Space developers,

We'd like to bring to your attention a feature coming to ESI very soon™ which may impact your application, depending on your use case.

As you all know, ESI does not have any rate limits. This is great for pulling down large quantities of data over a short period of time, and is something that we work hard to ensure we can keep in place. Generally speaking, we're keen to support all use cases for ESI, so long as they do not impact its use for anyone else or the performance of the EVE cluster.

In pursuit of this, we are soon going to add error limits. It's generally more expensive to complete a failed request than it is a successful one due to extra logging and/or exception capturing. Additionally, it's very common for us to see applications hit runaway conditions where they relentlessly retry on endpoints with queries that will never return a valid response. This needlessly consumes resources that could be put to better use with a bit of careful targetting.

What you may have noticed over the last week or so are some extra headers in your ESI responses. Notably, the X-ESI-Error-Limit-Remain header, which indicates how many errors responses will be returned to you in the current error window, and the X-ESI-Error-Limit-Reset header, which indicates the number of seconds until the end of the current error window.

For the moment, when X-ESI-Error-Limit-Remain reaches zero you will receive a warning header of X-ESI-Error-Limited. Once we enable this feature, you will instead recieve a 420 error response with a message encouraging more thoughtful use of resources. Once the error limit is breached, all of your queries will 420 error across all endpoints for the remainder of the error window, immediately and regardless of whether they would have succeeded or not.

Every response that is 4xx or 5xx level is counted against your limit. This includes timeout errors from ESI, due to EVE downtime or otherwise. At the current moment, we are targeting 100 allowed errors per 60 second windows.

Now would be an excellent time to add logging to your applications to look for the above mentioned headers, and to implement error-retry-backoffs if you haven't yet.

As always, to discuss this feature or anything else related to ESI, please join us in the #esi channel on tweetfleet slack. If you're not on tweetfleet slack yet, you can get an invite here -> https://www.fuzzwork.co.uk/tweetfleet-slack-invites/

With love,

-Team Tech Co

]]>
error limiting imminent error-limiting-imminent
http://developers.eveonline.com/blog/article/universe-names-update/ /universe/names Breaking Update 2017-08-25T15:17:48Z 2017-08-25T15:17:48Z Team Tech Co 95870245 Hi all,

As you know, the /universe/names/ endpoint was taken offline last week. This action was taken due to a new traffic profile to this endpoint that was causing disproportionate load on the database. It turns out this has been a lurking flaw in the endpoint since it was released, and it's just not been used this way until now.

Tech Co is developing technology to help the EVE server defend itself better against unforeseen load issues like this without requiring rate limits, but that technology is at least 4 weeks away from being ready. We will be releasing a blog explaining how this system will work closer to the deployment date.

Over the last week, we've been working the problem to find a way to bring this endpoint back earlier than that date. The /character/affiliations/ endpoint has the same flaw, and the changes below will be implemented on that endpoint as well.

Currently, these endpoints accept a list of IDs that may or may not be valid entities, and return the subset of them that have valid data. Going forward, these endpoints will return 404 if any single entry in the batch is not a valid ID. This change will be permanent on both endpoints.

Additionally, we are going to temporarily supplement the ability of these endpoints to detect invalid IDs without contacting the monolith, by using a dirty hack. Prior to initiating a round-trip to the DB with your query, we will test it against a set of hard-coded values that will filter out the vast majority of the values in the int32 range that are not valid for these endpoints.

Unsurprisingly, this hard-coded approach is a flawed half-solution. It will be better than nothing until we finish our work on the monolith load management code, but it has two key weaknesses you need to know about.

First, we have deliberately excluded dust characters from the hard coded values. This halves the number of valid IDs, and we consider it a worthwhile trade-off to re-enforce this temporary solution. Dust characters can still be resolved one at a time via the /characters/{character_id}/ endpoint, and their affiliations determined by spidering up the similar corporation and alliance endpoints.

Second, there is a set of very old character and corporation ID's that exists in an extremely large legacy range. This range is approximately 2 billion ID's wide, which means that including it would nullify the protective value of this technique. Characters and corporations with IDs between 100000000 and 2099999999 should be resolved individually against the character and corporation endpoints until we can implement a more permanent solution.

It is our hope that these techniques will be sufficient to tide us over to a better place, but the health of the EVE cluster will come first. If this approach is insufficient, we will fall back to offlining the endpoints again. If that happens, both endpoints will remain offline until the load management code is deployed.

With these compromises in place, the /universe/names/ endpoint will be onlined again shortly, and the /characters/affiliations/ endpoint will be updated at the same time.

As always, we're available to talk in the #esi channel of the tweetfleet slack if you've got questions.

-Team Tech Co.

]]>
universe names update universe-names-update
http://developers.eveonline.com/blog/article/2017-08-18-esi-outage-1/ ESI Outage: 2017-08-18 2017-08-22T17:58:52Z 2017-08-22T17:58:52Z CCP Logibro 93608389 Hi all

UPDATE: The issue with the locations and wallet endpoints has been resolved, and those endpoints are now online again.

ESI had a general outage yesterday between 14:15UTC and 21:00UTC. The locations and wallet endpoints remain offline to triage the underlying issue, which has been identified but requires further investigation. They will remain offline over the weekend.

The endpoint /universe/names has also been offlined due to to an unrelated issue.

None of these endpoints are deprecated. All will be returning ASAP, once the issues have been resolved. As always, we are available to discuss the situtation in the #esi channel of the tweetfleet slack.

More as it happens,

Team Tech co.

]]>
2017-08-18 ESI Issues 2017-08-18-esi-outage-1
http://developers.eveonline.com/blog/article/esi-open-sourced-front-end/ Setting up and Running esi-swagger-ui: ESI's Newly Open Sourced Front End 2017-08-08T14:20:07Z 2017-08-08T14:20:07Z CCP Zoetrope 2112625428 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


On July 11th, Team Tech Co open sourced the repository behind ESI's user interface: esi-swagger-ui. In doing so, we wanted to provide a way to the community to have documentation available offline as well as giving the community a chance to contribute and make ESI better looking and easier to use.

esi-swagger-ui is a fork of the official swagger-ui project and is built on version 3 of that project. At the time of this writing, the default view for ESI is still using swagger-ui version 2 at https://esi.tech.ccp.is, but you can see a live running instance of the ESI documentation using swagger-ui version 3 at https://esi.tech.ccp.is/ui/. This will be the case until version 3 has a few bugs worked out.

This blog will focus on getting set up and running esi-swagger-ui locally as well as offering a few development tips. Because a big step in setting up the repository is dependent on bash then it is assumed the reader is running a Unix based system. However, running the UI itself is done using Docker so a Windows user could manually do the steps found in run.sh. In fact, feel free to make the setup process easier for Windows users and make a pull request!

Prerequisites

  • Docker needs to be installed and running. Look here for more info on what Docker is.
  • Git so that you can clone the repository.
  • A Github account.

Setting up your environment to run esi-swagger-ui

There are four things we are going to walk you through that you will need to run esi-swagger-ui:

  • You need to know the IP address or URL your Docker daemon uses and set the environment variable DOCKER_MACHINE_NAME to that value.
  • You need an EVE third party application with all scopes enabled and the callback URL http://{your-docker-host}:8000/ui/oauth2-redirect.html where {your-docker-host} is the IP or URL for your Docker daemon.
  • You need a fork of the esi-swagger-ui Github repository cloned to your workstation.
  • You need a local file in the same repo named client_id that contains your third party application's client ID.

Finding and setting Docker's host

The following instructions were tested on Ubuntu 16.04

Finding Docker's IP

If Docker was set up correctly there should be a network interface named docker0 that will show you what IP Docker is using. Try running the command

ifconfig docker0

Look for inet addr, the IP address listed after is the docker host. A common CIDR for Docker is 172.17.0.0/32.

To give Docker a host name, edit your /etc/hosts file and append the following line (except using your Docker's IP address):

172.17.0.1  docker.local

After saving and exiting you now should be able to access the Docker daemon by using docker.local. Quickly test that this is the case by doing

ping docker.local

Setting DOCKER_MACHINE_NAME

Now that you know Docker's IP or have set it to be docker.local it is now time to set the environment variable DOCKER_MACHINE_NAME. Use the following command to set this:

export DOCKER_MACHINE_NAME=docker.local

Replace docker.local with Docker's IP if you did not set Docker's hostname to docker.local. If you want DOCKER_MACHINE_NAME to be available all the time, place the previous export command into your .bashrc or equivalent if not using bash and then refresh your session by logging out and in.

To see that DOCKER_MACHINE_NAME was set correctly use the following command:

echo $DOCKER_MACHINE_NAME

and you should see Docker's host or IP address displayed.

Creating a third party application with all scopes and the correct callback URL

  1. Go to https://developers.eveonline.com and login.
  2. Go to applications and click "Create new application".
  3. After putting a name and description, in "Connection Type" choose "Authentication & API Access".
  4. Have a clicking party with the scopes until they're all in the "Requested Scopes List".
  5. For the callback URL you will have to remember the Docker host from the previous section. Given Docker host docker.local the callback URL should be http://docker.local:8000/ui/oauth2-redirect.html. Replace docker.local with whatever your Docker host is.
  6. Click "Create Application".

Now you should have a third party application that uses all scopes.

Forking and cloning the repository

Navigate to https://github.com/ccpgames/esi-swagger-ui in your browser and click on "Fork" in the upper right corner. When asked where the repository should be forked to, click on your Github account. Once the forking has been completed you should be looking at a web page with your forked repository. You'll know it's yours if the name of the repository has your github username in it (for instance mine is ccp-zoetrope/esi-swagger-ui).

After you've done that follow this guide and do steps 2 and 3 to get your fork cloned locally onto your work station and also synced with the original ccpgames/esi-swagger-ui repo.

Using your client ID with esi-swagger-ui

With https://developers.eveonline.com/applications open, find the application you created previously and click on "View Application". Find the client ID and copy it to the clipboard.

Go back to your terminal and make sure you are in the root of the forked repository you just cloned. Once there, you need to put your client ID into a file called client_id (there should be no filetype ending). Do this by using a text editor or running the following command:

echo "{your-client-id}" > client_id

where {your-client-id} should be replaced by your application's client ID. Once that's done you should be ready to run everything!

Running esi-swagger-api

Still in the root of your forked repository, do the following command:

./run.sh

and it will do the following things:

  1. Fetch the latest, dev and legacy ESI Swagger specs and store them on your local machine under the directories latest/, legacy/ and dev/.
  2. Build a Docker image that uses your client ID.
  3. Stop the current running esi-swagger-ui Docker container if it is running.
  4. Run the newly built image inside a Docker container.

If you want to know more details about what is happening then feel free to dig into the files run.sh and Dockerfile.

To check if it is successfully running, do the command

docker ps

and you should see output similar to:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
d8dc8c278619        esi-swagger-ui-v3   "sh /usr/share/ngi..."   9 minutes ago       Up 9 minutes        0.0.0.0:8000->8080/tcp   esi-swagger-ui-v3

Now, go to your browser and navigate to http://{your-docker-host}:8000/ui and you should see the ESI documentation appearing before your eyes.

Updating to the latest ESI spec

This version of ESI's UI is not dynamically generated like the UI on https://esi.tech.ccp.is but instead downloads a local copy of the latest, dev and legacy specs. Because of this you need to manually update these specs to see the latest changes. To do that, simply delete the latest/, dev/ and legacy/ directories that were created when you ran esi-swagger-ui and re run it using run.sh. This should re-download the most up to date Swagger spec for ESI.

Contributing

Now that you have ESI's UI running locally you can start hacking away and contribute! Here in Team Tech Co we follow the "Fork-and-Branch" Git workflow for contributions. Here is a detailed walkthrough of this Git workflow that should help you contribute. If you've never contributed to an open source project before here is a great rundown of how to actually contribute to open source projects. If you have an idea for how to improve the ESI UI then please don't hesitate to contribute!

To make sure your time is well spent please open an issue in the esi-swagger-ui Github project to discuss the changes you would like to make before actually implementing anything. This way you can know if we also agree with your proposed changes and would like you to move forward with them. Changes submitted to the repository will also need to be accompanied by a signed Contributors License Agreement and we will arrange to have these dispatched to contributors as the final stage of approving a pull request. Our CLA looks like this, but may be subject to change.

Appendix

Since the last blog, the following XML API endpoints have been ported to ESI:

category XML endpoint ESI parity
char    
. CharacterSheet https://esi.tech.ccp.is/latest/#!/Character/get_characters_character_id_fatigue (This was added on top of what's already there for parity with the XML endpoint)
. Clones https://esi.tech.ccp.is/latest/#!/Clones/get_characters_character_id_implants
eve    
. FacWarStats https://esi.tech.ccp.is/latest/#!/Faction32Warfare/get_fw_wars
https://esi.tech.ccp.is/latest/#!/Faction32Warfare/get_fw_stats

The following ESI endpoint's versions have been bumped since the last blog:

  • GET /latest/corporations/{corporation_id}/alliancehistory/ (v2)
  • GET /dev/characters/{character_id}/clones (v3)
  • GET /dev/universe/types/{type_id} (v3)
  • GET /latest/universe/system_kills/ (v2)
  • GET /latest/dogma/effects/{effect_id} (v2)
  • GET /latest/characters/{character_id}/planets/{planet_id}/ (v3)

Complete and up-to-date list can be found here.

o7

CCP Zoetrope

]]>
ESI Open Sourced Front End esi-open-sourced-front-end
http://developers.eveonline.com/blog/article/swagger-codegen-update/ ESI Swagger Codegen Issues and Workarounds 2017-07-21T13:11:28Z 2017-07-21T13:11:28Z CCP Zoetrope 2112625428 This blog addresses an inconsistency [1] with a tool used within our previous blog: Using Swagger Codegen to Generate a Client API.

In the previous blog we chronicled how to generate an ESI client library for Python using the Swagger Editor or curling the https://generate.swagger.io endpoint (which the Swagger Editor uses). However, it's been brought to our attention that https://generate.swagger.io is not working with the ESI Swagger spec found at https://esi.tech.ccp.is/_latest/swagger.json.

Because online Swagger tools may not always work, this blog will present a backup method that can be used to generate a client library from ESI's Swagger spec.

Prerequisites

For this specific workaround you will need Docker installed on your machine. We will not go into detail about what Docker is or how it works but feel free to dig into Docker's documentation.

Generating a Python Client Library Using Docker

After installing Docker, run the following command:

docker run --rm -v ${PWD}:/local swaggerapi/swagger-codegen-cli generate -i "https://esi.tech.ccp.is/_latest/swagger.json" -l python -o /local/python-client

which will produce a directory called python-client with the client library inside. python-client will be situated wherever the docker command was run.

What this command is doing is pulling Swagger's Docker image called swaggerapi/swagger-codegen-cli, running the image inside a Docker container and then deleting that container once it is done.

In Unix based systems, generating the python-client may create some permission problems when trying to follow the directions for using the library in our previous blog. For good measure, run the following command on the python-client directory:

sudo chown -R $USER python-client

Generating a Python Client Library Without Docker

Instructions for building and running the swagger-codegen-cli outside of Docker can be found on the swagger/swagger-codegen Github page.

[1] Somewhere around July 14th, 2017 the code behind https://generator.swagger.io was updated with Swagger codegen version 2.2.3. This version of the Swagger codegen has a bug in its parser that is making the generator not work for the ESI Swagger spec. According to a Swagger dev their online tools will be updated end of August 2017 with Swagger codegen 2.3.0 which is the version that comes with the Docker image used in this blog.

o7

  • CCP Zoetrope
]]>
Swagger Codegen Update swagger-codegen-update
http://developers.eveonline.com/blog/article/esi-and-optional-booleans/ ESI and optional booleans 2017-06-26T16:34:45Z 2017-06-26T16:34:45Z CCP SnowedIn 95538921 Greetings space developers,

Just wanted to send out a short update to notify everyone of a behavioral change in ESI. By the time you're reading this, ESI will have stopped sending optional boolean attributes when they are not True.

If a spec has the boolean as required, the value will always be sent. Otherwise the key/value will only be sent if the value is True.

An object property is required only if it is listed in the object schema's required array. For example, the alliance object in the GET /v1/corporations/{corporation_id}/alliancehistory/ 200 response array, each object has an alliance property, which is defined as:

{
  "description": "alliance object",
  "properties": {
    "alliance_id": {
      "description": "alliance_id integer",
      "format": "int32",
      "title": "get_corporations_corporation_id_alliancehistory_alliance_id",
      "type": "integer"
    },
    "is_deleted": {
      "description": "True if the alliance has been deleted",
      "title": "get_corporations_corporation_id_alliancehistory_is_deleted",
      "type": "boolean"
    }
  },
  "required": [
    "alliance_id",
    "is_deleted"
  ],
  "title": "get_corporations_corporation_id_alliancehistory_alliance",
  "type": "object"
}

By listing is_deleted in required ESI will return the is_deleted property regardless of its value. In contrast, the GET /v2/corporations/{corporation_id}/alliancehistory/ 200 response array is defined with:

{
  "description": "200 ok object",
  "properties": {
    "alliance_id": {
      "description": "alliance_id integer",
      "format": "int32",
      "title": "get_corporations_corporation_id_alliancehistory_alliance_id",
      "type": "integer"
    },
    "is_deleted": {
      "description": "True if the alliance has been closed",
      "title": "get_corporations_corporation_id_alliancehistory_is_deleted",
      "type": "boolean"
    },
    "record_id": {
      "description": "An incrementing ID that can be used to canonically establish order of records in cases where dates may be ambiguous",
      "format": "int32",
      "title": "get_corporations_corporation_id_alliancehistory_record_id",
      "type": "integer"
    },
    "start_date": {
      "description": "start_date string",
      "format": "date-time",
      "title": "get_corporations_corporation_id_alliancehistory_start_date",
      "type": "string"
    }
  },
  "required": [
    "start_date",
    "record_id"
  ],
  "title": "get_corporations_corporation_id_alliancehistory_200_ok",
  "type": "object"
}

The response has been flattened, what used to be a nested attribute is now part of the base object. However, is_deleted is not in the required array. Because of this, if the alliance is not deleted, the is_deleted parameter will not be included.

Depending on your language and the swagger client you are using, you may notice that you have to access parameters in a safer way (ie, use resp.get("some_key") instead of resp["some_key"] in python). This will be very implementation specific though, consult the documentation for your swagger client and/or language of choice.

-CCP SnowedIn

]]>
ESI and Optional Booleans esi-and-optional-booleans
http://developers.eveonline.com/blog/article/swagger-codegen/ ESI Tutorial: Using Swagger Codegen to Generate a Client API 2017-06-08T00:00:00Z 2017-06-08T00:00:00Z CCP Zoetrope 2112625428 This blog post is part of a series of blogs examining best practices for ESI development. Each blog will be published on the 8th of each month during the journey towards XML API and CREST’s termination date. The legacy APIs will be terminated on May 8th, 2018, or earlier if metrics signal a trivial level of usage.


Here on this blog we regularly mention the fact that ESI uses a framework called Swagger. In fact, the word "Swagger" is directly in the name (EVE Swagger Interface)! We also regularly mention that Swagger provides the capability of auto generating code for the programming language of your choice. However, it's not entirely obvious from looking at Swagger documentation how one should go about generating and using a client library. This blog will cover two ways to generate a library using ESI with tools provided by Swagger.

Method 1: Using the Swagger Editor

Note: If you have problems using the Swagger Editor, look at this blog.

For those preferring UIs to bash, this is definitely the easiest way to go about library generation using Swagger. First, navigate to https://editor.swagger.io which will display an online editor window split into two panes. The pane on the left is for an OpenAPI specification while the pane on the right will show an interactive interface using that spec. Load up the ESI spec by clicking on the "File" drop down at the top and then clicking on "Import URL":

Swagger editor import URL

The latest versioned Swagger spec for ESI is available from the URL https://esi.tech.ccp.is/_latest/swagger.json. Drop the URL into the dialog that appears after clicking "Import URL" and the left pane will populate with the ESI Swagger spec. Look here for details on why the above URL is recommended.

Click on the "Generate Client" dropdown and, for the purpose of this tutorial, pick python (we all know flash is the best though /s).

Swagger editor import URL

This will download a zip file containing a generated client for Python. Extract this anywhere you would like so you can work with it.

Method 2: Curl'n It Up

Note: If you have problems using the Swagger Editor, look at this blog.

This method can only be used on Unix based systems and is for those wanting to automate all the things.

Swagger supplies an api that allows the programmatic generation of client libraries. The spec for the generator API itself (generated using Swagger of course) can be found at http://generator.swagger.io. The gist is this: send a POST request to the /gen/clients/{language} endpoint (replace {language} with the language of your choice) with a payload containing a URL pointing to the ESI Swagger spec. A curl command like the this will do the trick:

            curl -X POST -H "Content-Type: application/json" -d '{"swaggerUrl":"https://esi.tech.ccp.is/_latest/swagger.json"}' "http://generator.swagger.io/api/gen/clients/python"
        

and the API will return a response that looks like

            {
              "code": "d40029be-eda6-4d62-b1ef-d05e2e91a72a",
              "link": "http://generator.swagger.io:80/api/gen/download/d40029be-eda6-4d62-b1ef-d05e2e91a72a"
            }
        

The link in the returned object can then be used to download a zipped client library with a tool like wget.

 

Using the Generated Python Client

In the terminal/command prompt navigate to the directory where you unzipped the python client. If the unzipped directory was not renamed, it should be called python-client. Create a python virtual environment using virtualenv with the command

            virtualenv venv
        

and then activate the virtual environment:

 

For Unix based systems use source venv/bin/activate.

For Windows systems use venv\Scripts\activate.

Once inside the virtual environment, install the dependencies required for the Python client by running

            pip install -r requirements.txt
        

requirements.txt can be found at the base of the python-client directory. To test it out, drop into a python shell by typing

            python
        

then do the following:

            import swagger_client
            from swagger_client.rest import ApiException

            api = swagger_client.CharacterApi()
            api.api_client.set_default_header('User-Agent', 'my-test-agent') # Set a relevant user agent so we know which software is actually using ESI
            api.api_client.host = "https://esi.tech.ccp.is" # see [1]

            try:
                response = api.get_characters_character_id(2112625428)
                print(response)
            except  ApiException as e:
                print("Exception when calling CharacterApi->get_characters_character_id: %s\n" % e)
        

If everything went as planned then the response should contain details about the character CCP Zoetrope.

To figure out how to call other ESI endpoints read python-client/README.md and look at the documentation in python-client/docs as it has code examples for calling every endpoint.

[1] This is needed to bypass an issue with Swagger code generation. The Swagger specification does not expect base paths that are a single forward slash "/", and the generated code currently contains an extra forward slash in the resulting full path. ESI uses strict URL matching, so this causes 404s. We will work with the Swagger community to fix this in the code generation stage if we can. For the time being this is a necessary step when using underscore routes.

Conclusion

As has been shown, it's relatively simple to get an auto generated ESI client library for most languages and frameworks. Simply pick the language you want to develop in, get generated ESI code using one of the methods above and get crackin' on that dream EVE Online flash plugin you've always wanted.

o7

- CCP Zoetrope

Appendix

The following paths have been added or updated in ESI since the previous blog:

  • GET /latest/characters/{character_id}/industry/jobs (v1)
  • GET /latest/characters/{character_id}/orders (v1)
  • GET /latest/characters/{character_id}/blueprints (v1)
  • GET /dev/dogma/effects/{effect_id} (v2)

Get a sneak peek about what's coming to ESI by watching this board.

]]>
Swagger Codegen swagger-codegen
http://developers.eveonline.com/blog/article/accountstatus-xml-deprecated/ Caution: AccountStatus.xml.aspx Deprecated and Unreliable 2017-05-26T16:50:53Z 2017-05-26T16:50:53Z Team Tech Co 95870245 Currently, the XML API endpoint /account/AccountStatus.xml.aspx is returning unreliable data. This endpoint has been officially deprecated since November 2016. Team Tech Co. has decided that this issue is not considered a "critical bug". As such, there are no plans to make changes to the endpoint to fix this issue (per our announcement here). The reasons we did not class it as such are:

  • Account level information will not be available in ESI.
  • Players receive notification e-mails directly from CCP if their non-recurring subscription is approaching expiry.
  • This endpoint's original intent is no longer relevant, as players are not prevented from logging into EVE if their Omega status lapses.

Despite being deprecated and unreliable, we will be leaving the endpoint active so as to not disrupt applications that would break if it was removed. We strongly caution against relying on this endpoint. We again remind third party developers that this information will not be provided to third party developers upon termination of the XML API, and that you should be removing all usage of the XML API from your applications going forward.

]]>
AccountStatus XML Deprecated accountstatus-xml-deprecated
http://developers.eveonline.com/blog/article/changes-to-public-citadel-endpoint/ Changes to the public citadel endpoint 2017-05-05T15:07:26Z 2017-05-05T15:07:26Z CCP Bartender 95465499 Greetings all!

On Tuesday 9th of May, Team Tech Co will be releasing a behavior change to the /universe/structures/ endpoint. This change will affect all versions of the endpoint, and although it will not change the specification of the endpoint (so you won't need to update your code), you may wish to know about how the behavior changed and why.

Until now, this endpoint has been returning the structure ID’s of every structure that had the "public" entity on its ACL's allowed docking rights. This was done on the basis that such citadels are clearly visible in space and in the in-game structure browser, and thus matched information given in game.

However, this is not quite correct. It is possible to add the "public" entity to one's ACL's, and then explicitly ban someone. Some of you may have noticed that if you crawl all structures on the public list, some of them will return access denied errors despite ostensibly being "public". That happens because you, your corporation or your alliance is explicitly banned from that citadel.

As such, we have until now been returning structures that are kinda-sorta not public. We don't consider this to be an information leak, because these structures are easily discovered and regularly indexed by players the old fashioned way using armies of alts parked in every region in the game anyway, but it is damn confusing when you're trying to use the API, and unintuitive to the owners of the citadel.

We've modified this behavior, and from 2017-05-09 structures will only show up in the public structures API endpoint if they have the public entity on their docking list and also have not banned anyone from docking. If you wish for your citadel to be public and discoverable, you will need to meet these criteria, otherwise your citadel will be considered selective, and will not be discoverable via the API.

Team Tech Co is available in #devfleet on the Tweetfleet Slack if you have any further questions

-CCP Bartender

]]>
Changes to Public Citadel Endpoint changes-to-public-citadel-endpoint
http://developers.eveonline.com/blog/article/breaking-changes-and-you/ Breaking changes and you - How to use alt-routes to enhance your sanity 2017-02-01T14:19:39Z 2017-02-01T14:19:39Z CCP Bartender 95465499 Afternoon developers!

We've mentioned it before, but I'd like to give everyone a clear best-practices guide to how to use alt-routes in ESI to maintain application stability against a constantly evolving API.

Schedule of breaking changes

Before I get into that though, we previously said we'd be updating a forum thread with all planned breaking changes to ESI. That's turned out to be a bad solution to letting you all know when we're planning to change stuff. It's hard for you to read at a glance, and it's totally outside the tools we use day-to-day to develop ESI.

We've moved this information to the ESI deployment timeline, here: https://github.com/ccpgames/esi-issues/projects/2.

The timeline shows the deployment of totally new routes, but it also shows the deployment of routes that are going to have breaking changes. Consider the following screenshot:

We can see that on the 24th of January, the following happened:

  1. The /dev/characters/{character_id}/ route was bumped to v4. This indicates a breaking change on this route.
  2. /latest/wars stayed on v1, but we're making a breaking change to the spec.[1]
  3. /latest/universe/races was bumped to v1. Since v0's are not a thing, that means this was a newly created endpoint.

In general, you should know what versions of what routes you are using, and keep an eye on this page so you can see if those routes are going to be altered.

The importance of alt-routes

I've mentioned /v1 here, but let me expand, because this is important.

If you look at the top of the ESI UI, you will see three buttons: "legacy", "latest" and "dev". When you click on one, it will show you a list of endpoints that are available on that route. For example, you can make a curl request to https://esi.tech.ccp.is/latest/characters/95465499/, and get data about my marvelous self. Notice the "latest" in the middle of the URL.

You can also make requests to https://esi.tech.ccp.is/dev/characters/95465499/ (dev) and https://esi.tech.ccp.is/legacy/characters/95465499/ (legacy). These three routes may or may not return different data. Team Tech Co is supporting exactly three versions of a route at any one time:

  1. /dev (where we may test changes with the community before promoting them, if appropriate. There is no stability here, we'll break this one without hesitation.)
  2. /latest (the one you should be using)
  3. /legacy (the previous version of latest, for you to use until you can update your app back to current latest)

These three routes (legacy, latest, dev) are convenience labels to help you understand what is "current" at any given time. You should not be using them in your source code, as you will see in a minute.

When Tech Co promotes a change, we take whatever is in /legacy and throw it away. We take whatever was in /latest, and put it in /legacy, and put /dev in /latest. However, there's a problem here. If you're calling the /latest route in your app and we promote a breaking change, the /latest route will change underneath you, and your app will break. The legacy route still has the version you currently rely on, but your app is broken anyway. Sadface.

 

This is why you should be using versioned alt-routes. if you take a look at the current latest of /characters/{character_id}, you will see at the top a set of alternate routes: 

As you can see from the screenshot above, the current alt-routes are /v4 and /dev. These routes are aliases for /latest. This aliasing goes multiple directions. You can infer that /latest and /dev are identical at the moment, because /dev is an alt-route of /latest. /latest is also an alt-route of /dev, and behind the scenes all three are being directed to the same block of code.

However, more interesting is /v4. If you make a request to /v4, you will get the same as /latest. When we next promote the /characters/{character_id} route, we will move /v4 to be an alias of legacy. This means when we promote a breaking change your app will not break if you are using /v4. Instead, you will be smoothly transitioned into /legacy.

Once an alt-route is only available on /legacy (or if you call /legacy directly), your app will start receiving "warning" headers with every payload, even on 401 unauthorized errors. If your app is running through a CI system, one of your tests should be "blindly call every route I use and see if I get a warning header in the return". This will tell you if any of the routes you are using have been moved to legacy, so you can update your app back to latest.

So, to recap, to avoid having your app suddenly break when we push breaking changes to endpoints:

  1. Use the versioned alt-routes for everything.
  2. Never use the convenience label routes in your source code.
  3. Have some kind of test you can run that will aggregate and call all the routes your app uses, and tell you if it gets warning headers.
  4. Keep an eye on the deployment timeline!

And of course, the ever present 5:

If you want to keep up to date with ESI development or ask interesting questions, the best place to do it is in the #esi and #devfleet channels of the tweetfleet slack. You can request a tweetfleet slack login here.

 

I hope that explains a few things!

Bartender out~~


[1] This is complex, make sure you've read the rest of the blog before you come back to this. The code backing the /v1 route of wars was causing performance problem with the TQ cluster, so we chose to change the endpoint without bumping the version. This bypasses all the best practice stuff I just finished telling you, and risks breaking peoples apps. Because the stability and performance of Tranquility trumps other concerns, we chose to take that risk here. We couldn't afford to have people continuing to use the broken code, so we couldn't move it to legacy. Tricky problems with no good solution like this are exactly why it's important for you to keep an eye on the schedule and idle in the #ESI if you're running anything mission critical. This is not SOP; we only do stuff like this if we're backed into a corner.

]]>
Breaking changes and you - how to use alt-routes to enhance your sanity breaking-changes-and-you
http://developers.eveonline.com/blog/article/esi-mail-endpoints/ ESI Mail Scopes Are Now Released 2017-01-24T17:57:46Z 2017-01-24T17:57:46Z CCP Bartender 95465499 Greetings developers!

Team Tech Co. is pleased to announce that all mails scopes (esi-mail.organize_mail.v1, esi-mail.read_mail.v1, and esi-mail.send_mail.v1) have now been released and are available for general use. We're excited to see what everyone does with them! (Thunderbird plugin, I want to believe!)

However, I'd like to drop a quick reminder about section 2.3 of the EVE developer license agreement: 'Developer acknowledges and agrees that no Application shall be used [...] (b) as a means of phishing or spamming'

Our third party developers have great judgement when it comes to these types of issues, but I wish to make it clear that one of our key indicators of "spam" will be "whether other people think you are sending spam". Applications that create work for our GM's by sending large volumes of unwanted mail (thus prompting people to create tickets asking us to do something about it) will not be tolerated. I would encourage you all to seek explicit consent to receive mail (above and beyond authorizing scopes) from your applications users, and to empower them to revoke that consent easily if they desire to do so. If you would like further guidelines, especially on the topic of dispatching mail to players who are not users of your application, The Spamhaus Project has a great page on the definition of spam.

Additionally, programmatic use of EVE Gate to dispatch mail is now considered a misuse of the EVE Gate service. We will provide a short grace period until the 7th of February to allow anyone using this mechanism to disable it. After this date, there will be consequences for anyone continuing to use this method to send EVE mails.

Let's all be good netizens, and happy developing!

~CCP Bartender

]]>
ESI Mail Endpoints esi-mail-endpoints
http://developers.eveonline.com/blog/article/character-id-range-change/ Technical Note: Character ID Ranges 2016-12-07T15:31:25Z 2016-12-07T15:31:25Z CCP Bartender 95465499 Greetings developers,

Quick info dump here. Up until now, all character ID's have been totally sequential and existed in rigidly defined ID ranges. In support of the influx of new players from the Ascension release, we are laying the groundwork to move characters into a non-sequential range (much like citadel ID's). More news on that will come next year (no need to panic ^_^), but for right now, you should know that all new characters on singularity are being created with ID's starting from 2112000000. This change will be released on TQ next Tuesday, 13th Dec 2016.

For the time being, these new character ID's will remain sequential. However, CCP makes no guarantees that this will continue to be the case, and caution against encoding that assumption into your application logic as a long term strategy.

(And yes, once we're closer to the time, we'll be looking at endpoints to allow you to discover new characters, I know a few of you need a way to do that).

Bartender out <3

]]>
Character ID Range Change character-id-range-change
http://developers.eveonline.com/blog/article/introducing-the-esi-api/ Introducing the ESI API 2016-11-08T18:45:20Z 2016-11-08T18:45:20Z Team Tech Co 95870245 The EVE Swagger Interface is a new framework developed by Team Tech Co to underpin the new ESI API. If you don't know what this is, you should go here to read the blog Introducing ESI - A new API for EVE Online.

This blog will dive a little deeper into the design and technical specifics of ESI, going to get into the base design of ESI to explain a few basic principles of what's going on behind the scenes. Most of this is transparent to the front-end, but may be of interest to some capsuleers regardless.

Fundamentals

Multi-tenant to the core

ESI was designed to be a single interface to all information EVE related. As such, if ESI and the cluster agree on configuration, you can change the datasource query parameter on any route (including /<version>/swagger.json) to the server name of your choice.

You will see this in the ESI UI as a select menu in the header/nav bar, and in the datasource enum in the spec itself.

As a (potentially) interesting side-note here, there are actually three ESI environments running, an esi-dev, esi-test, and the production esi. With esi-dev or esi-test and a bit of configuration (which will be replaced with tooling), EVE developers can actually query their local development server through ESI.

Authentication

Auth is still handled by SSO. There are new scopes which ESI uses, you can make new developer keys (or alter your existing) to make use of the new scopes. If an ESI endpoint requires authentication, you will see a red exclamation mark on the route description in the swagger UI. Mouse hovering over this icon will tell you what scope the endpoint requires.

ESI will handle redirecting your authentication header to the correct SSO for verification (depending on the datasource query string argument). SSO is not multi-tenant though, so you will need to create app keys on each SSO backend you intend to use.

Versioning

ESI itself will have an API version (as defined in the spec), that version number however, is mostly irrelevant to API consumers (it's the version of ESI-lib). Instead, ESI versions each route individually. This allows us to make much faster changes and avoid the awkward situation of global API versioning.

As an example, let's say you have a route /v1/hello/<string>/, and you want to change the path parameter to accepting an integer instead. With a traditional API, the entire APIs basePath would have to be bumped. This is obviously a little less than ideal, considering there could be hundreds of other unchanged routes.

With ESI, you always (and only) get three complete APIs, /dev/, /latest/ and /legacy/. But each route is also given a numbered path, starting with /v1/. Alternate routes are mentioned in the route description (until something better comes along in the standards?).

/dev/ can change at any point, changes to /latest/ will be announced. After changes are made the previous /latest/ will be available as /legacy/, until the next version bump. If you want to avoid the return schema of your request suddenly changing, you can use the versioned alternate route. In that case, prudent developers may want to create unit tests (watch for the "warning: 199" headers) to notify them when endpoints are moved to legacy. Additionally, we will be updating a stickied thread in the EVE Technology Lab subforum as changes happen.

Additionally, endpoints may be deprecated. When an endpoint is deprecated, a line appears through it on the swagger UI, and it begins returning the "warning: 299" header. This is slightly different than a warning: 199 header, which you will receive if an endpoint was updated and there is now a newer version of it available. Deprecation is how Tech Co broadcasts an intent to delete a route. Deprecated endpoints may include a recommended alternate source or other message in the 299 warning, and you should move away from them immediately.

Because endpoints are versioned individually, the concept of an overall /v1/ (or 2 or 5) API is not very relevant, and no swagger-ui is provided for these routes. Browsing the API's capabilities should be done via /dev/, /latest/ or /legacy/. However, users who wish to indulge their curiosity may feed a /v1/swagger.json into their own Swagger UI to get an overview, if they wish.

As an aside, all ESI routes end with a /. The only exceptions are the /<version>/swagger.json routes and routes used by the swagger-ui, which are passed through ESI.

Rate limiting

There are no rate limits in place, the plan is to rely on caching more than rate limiting.

ESI returns standard caching headers if the data is cached. Applications should notice and make use of these headers (expires and last-modified) on routes where they are provided.

Decoupling

We briefly touched on this earlier, but one of the major motivating factors of this design was to decouple the API from the monolith itself. As a consumer of the API, you should not have to care at all about the database structure, how or where the data is retrieved. All you should need to know is what inputs are allowed/accepted, and what exactly is being returned.

ESI takes this one step further. There is a generic concept of a "data relay" inside ESI, built on top of Google's PubSub. Because of this decoupling of data and API, it allows the retrieving end to move out of the monolith gradually and transparently. As long as something somewhere is listening and responding to events via PubSub, ESI will not notice if the service leaves the monolith and becomes standalone.

Moar design!

In the ESI API system, the codebase is split into two separate yet equally important groups: the esi-routers, who proxy routes and aggregate specs; and the esi-[endpoint]s, who fulfill the requests. These are their designs.

ESI-router

The routers are where most of the magic happens. They are the ones integrating with the Kubernetes API. The router finds the endpoints, and from each one, requests all of its swagger.jsons. It then combines all of these partial specs into versioned global specs. It serves the global specs, proxies the swagger-uis, and proxies ESI routes to their appropriate endpoint.

The routers are also a cluster, and use Redis to elect a leader among themselves. The leader continually checks for changes to the distributed spec, and the other routers all check for changes on the leader.

All routers also maintain telemetry on the endpoints, measuring the latency of each call, as well as tracking who made the request.

ESI-[endpoint]s (there are many)

The endpoints can be considered mini, stand-alone ESIs. If you found a way past the router cluster and hit the endpoint directly, you'd find it serves a swagger.json for each version it has routes for (in addition to dev, latest and legacy), but does not have any other user interface. All authentication and caching happens on the endpoint level. Each endpoint serves one or more routes in its own little cluster (behind a Kubernetes ClusterIP service), able to horizontally scale at will.

Would you like to know more?

Tech Co developers are regularly available on the tweetfleet slack (@ccp_bartender, @ccp_snowedin, @ccp_aquarhead, @ccpchimichanga), and we will be monitoring Reddit and the EVE Online forum thread if you've got questions or comments. We're especially keen to talk about anything you need from us to port your existing applications over from the XML API, so please stop by for a chat!

To the glorious future!

•     Team Tech Co

]]>
Introducing the ESI API introducing-the-esi-api
http://developers.eveonline.com/blog/article/legacy-sde-discontinued/ Legacy SDE discontinued 2016-10-20T14:27:18Z 2016-10-20T14:27:18Z CCP Tellus 95633963 Greetings capsuleers!

Along with the release of the Citadel expansion several months back, we improved upon the Static Data Export (SDE) by converting the remaining SQL data to YAML files. To aid in the transition and to give 3rd party developers time to migrate over to the new format, we had provided SDE builds in the old format alongside the new version after each release.

We are discontinuing builds of the old SDE version builds. New SDE releases will be YAML only.

That's all! Have a nice day. :)

~~ CCP Tellus

]]>
Legacy SDE Discontinued legacy-sde-discontinued
http://developers.eveonline.com/blog/article/yc118.8-crest-features/ YC118.8 CREST Features 2016-09-02T17:22:49Z 2016-09-02T17:22:49Z CCP Bartender 95465499 Greetings all!

There are some new and resurrected features coming to CREST in the YC118.8 release.

New remote client UI feature: open mail template

Resource:
/characters/<characterID>/ui/openwindow/newmail/

As part of our ongoing work on smoothing the transition away from the IGB removal, the /characters/<characterID>/ui/openwindow/newmail/ resource has been added. This POST resource will take a subject and body, and optionally a list of recipients, and open a pre-filled mail window on the client, ready for the user to hit send.

There are a few caveats due to idiosyncrasies in the eve mail system:

  1. Corporations, alliances and mailing lists are all types of mailing groups. You may only send to one mailing group, at a time, so you may either fill out the corporationOrAllianceRecipient field, or the toMailingListId field, but not both. You may send to specific recipients in addition to a mailing group. You can retrieve mailing list ID's from the XML API.
  2. There is a maximum number of recipients you may specify, currently set to 50. If you try to send to more than 50, everything after the first 50 recipients will be ignored.
    This value may be lowered in the future if we have spam or load problems.
  3. CREST will attempt to detect if you request a recipient character ID for a character that has not yet been created, and quietly remove the character ID from your recipients list as part of request validation.  However, due to underlying weirdness in the mail system our ability to do that without causing excess load is a bit spotty. I've decided to take a low tolerance approach to this, and CREST may elect to drop all recipients instead of just the incorrect ones. Ideally, validate your character ID's before dispatching them.

Restored tournament resource features: match frame data and match static data

Resources:
/tournaments/<tournamentID>/series/<seriesID>/matches/<matchID>/realtime/<frameID>/
/tournaments/<tournamentID>/series/<seriesID>/matches/<matchID>/static/

Back in the day (by which I mean late 2013), CCP provided a neat-o set of endpoints that allowed CREST developers to request tick-by-tick gameplay recordings of Alliance Tournament matches, and create visualisation tools to reconstruct and play back AT matches. This functionality broke in early 2014, and has been unusable for two years. They're now back online and ready for use in Alliance Tournament XIV! You can also retrieve the replay data from tournaments that ran while the endpoint was non-operational, for your viewing pleasure.

-CCP Bartender

]]>
YC118.8 CREST Features yc118.8-crest-features
http://developers.eveonline.com/blog/article/reminder-no-scamming/ Reminder to Developers: No Scamming 2016-07-04T12:46:58Z 2016-07-04T12:46:58Z CCP Bartender 95465499 Hello Developers

We just wanted to remind you that using any of our tools that require you to sign the Developer License Agreement to scam other players is against the terms of the developer agreement. This includes the use of the SSO or other authed CREST endpoints. The relevant part is below

2. Grant of License

3. Developer acknowledges and agrees that no Application shall be used (a) to inject, deliver, or activate malware, worms, Trojan horses, or other bugs, (b) as a means of phishing or spamming, (c) as a means of tracking Player information or Player activity without the express knowledge and consent of such Player, (d) as a means to misappropriate a Player's in-game items or other information, or to otherwise cheat, scam, or defraud Players who subscribe to the Application, or (e) as a means of implementing a denial of service attack, distributed denial of service attack, or other method of disabling or denying access to EVE or the Licensed Materials. Notwithstanding anything to the contrary in this Agreement, Developer shall be liable to CCP for all resulting losses, expenses, costs of defense, costs of correction, and other damages (including without limitation damages for loss of business or profit, business interruption, or any other pecuniary loss) resulting from Developer's violation of this section.

Any developer found breaching this provision of the license will have action taken against them.

Regards

-CCP Bartender

]]>
Reminder: No Scamming reminder-no-scamming
http://developers.eveonline.com/blog/article/improvements-to-corporation-structures-crest-endpoint/ Improvements to CREST endpoint Corporation structures 2016-06-16T14:40:34Z 2016-06-16T14:40:34Z CCP Bartender 95465499 Hi all, CCP Bartender here!

A quick note about some upcoming changes in the corporation structures resource. This resource was my first introduction to creating crest handlers, and as part of a routine review of the crest logs I have found that unfortunately it was not up to scratch.

Two key issues have been found:

  1. Due to a caching bug, one hour after a user first requests their corporation structures the resource would start returning 500 errors.
  2. I typo'd the hell out of the resource route, and it was under the tree root /corporation/ rather than /corporations/. Note the lack of trailing 's'. As such, it is inconsistent with every other corporation resource.

In addition, it also wasn't discoverable because it wasn't linked to from anywhere, which is kind of a CREST faux pas.

The logs tell me (perhaps unsurprisingly given it's state) that very few people are currently using this resource, so with all the apologies in the world to my early adopters, I'm going to change the route to match the standard for /corporations/ resources.

The 500 caching bug has also been fixed, and I've added a very minimal corporations resource that just returns the structures resource link if you request a /corporations/<corporationID>/ resource, so the corporation structures resource is now discoverable.

This means that if you're currently using the /corporation/<corporation id>/structures/ resource, you'll need to update it to /corporations/<corporation id>/structures/.

This change is planned to go live on the 21st of June.

Thanks for your understanding if you're one of the few caught out by this.

]]>
Improvements to Corporation structures CREST endpoint improvements-to-corporation-structures-crest-endpoint
http://developers.eveonline.com/blog/article/crest-renovations-2016-06/ CREST Renovations for June 2016 2016-06-07T18:58:59Z 2016-06-07T18:58:59Z CCP FoxFour 92168909 Hello everyone,

With DUST officially shut down as of May 30th I took some time to go through CREST and cleanup a bunch of things. The primary objective of this work was to remove everything relating to DUST from CREST, or more specifically everything not used by third-party developers for EVE. However, since I was making a number of changes that were going to break things anyway, I decided I may as well make a few other changes that will undoubtedly break other things. There are also a few smaller changes that don't actually break anything sprinkled into the mix for good measure.

These changes will be deployed on June 14th. Hopefully this is enough time for everyone to prepare for the changes. They are all live on Singularity now so you can go ahead and test against that to make sure your code works.

  • Market orders in the all orders resource are now sorted by order ID
  • /types/<typeID:integerType>/ is now /inventory/types/<typeID:integerType>
  • /types/ is now /inventory/types/
  • Removed "motd" from the root API
  • Added "id" to the market group "parentGroup" attribute
  • The root API is now just version 5; all previous versions have been deleted
  • "serviceStatus" in the root API is now just a string value and not a dictionary of "server", "eve", and "dust" values
  • "userCounts" is now "userCount" and a single integer value instead of a dictionary of "eve" and "dust"
  • Removed every CREST resource that only dealt with DUST and not used by third-party developers for EVE
  • The character resource (/characters/<characterID:integerType>/) v3 no longer exists and a new v4 has been added
  • The "navigation" resource (/characters/<characterID:characterIdType>/navigation/waypoints/) for setting waypoints has moved to /characters/<characterID:characterIdType>/ui/autopilot/waypoints/
  • The waypoints link is now found under character -> ui -> waypoints along with showMarketDetails and showContract
  • Links to characters that used to include the capsuleer link no longer do as that resource has been removed
  • Fixed an issue with some types throwing an internal server error due to graphic IDs
  • The market history resource has changed from /market/<regionID:integerType>/types/<typeID:integerType>/history/ to /market/<regionID:integerType>/history/ and now takes a "type" parameter exactly the same way the market buy or sell order resources do
  • Regions now link to the market history resource

The biggest changes for most third-party developers will be the changes to the type routes and the market history route. The types change has been a long time coming so that it matches the groups and categories (/inventory/groups/ and /inventory/categories/). The market history change is an even longer coming change. I just didn't want to break everyone’s stuff but since I am breaking stuff anyway, now seemed like a good time to do it.

Just as a final note, I am really sorry about all the things this is about to break. It breaks a bunch of CREST conventions, it breaks a bunch of people’s applications, etc. I wish there was a better way I could get this done but for various reasons we really needed to take the "just rip the band-aid off" approach and just get them done.

CCP FoxFour

]]>
CREST Renovations 2016-06 crest-renovations-2016-06
http://developers.eveonline.com/blog/article/new-crest-resource-for-bulk-market-orders/ New CREST Resource for Bulk Market Orders 2016-05-23T14:38:15Z 2016-05-23T14:38:15Z CCP FoxFour 92168909 Hello everyone!

I wanted to inform you all about a new market resource in the CREST API. Until now, if you wanted to get all the market orders for Tranquility you had to request every type per region individually. That would generally be 50+ regions multiplied by upwards of 13,000 types. Even if it was just 13,000 types and 50 regions, that is 650,000 requests required to get all the market information. And if you wanted to get all the data in the 5-minute cache window, it would require almost 2,200 requests per second. While some of you came up with creative solutions to this problem such as prioritizing faster moving items, we decided we needed a better solution that was both easier to use and reduced load on our servers. As such, we have added a new market resource that is aimed at anyone who is trying to get all of the market data.

The new market resource can be found under /market/<regionID>/orders/all/ and returns all market orders for a given region paginated, with up to 10,000 results per page. For busy regions such as The Forge, you would expect around 30 pages of data based on the current trading volume. Here is a sample of the output:

{
    "totalCount_str": "306700",
    "pageCount": 31,
    "items": [
        {
            "buy": false,
            "issued": "2016-04-02T02:15:32",
            "price": 5716.55,
            "volumeEntered": 50000,
            "stationID": 60003760,
            "volume": 9756,
            "range": "region",
            "minVolume": 1,
            "duration": 90,
            "type": 21368,
            "id": 4386193414
        },
        ...
    ],
    "next": {
        "href": "https://api-sisi.testeveonline.com/market/10000002/orders/all/?page=2"
    },
    "totalCount": 306700,
    "pageCount_str": "31"
}

One of the things you will note about this resource is that we have trimmed down each order to be very slim. We don't include a link to the type, instead we supply just a type ID. We don't include the _str attributes. Everything that could be stripped out has been in order to reduce this resource response in size. Be warned though, even in its stripped-down form each page is several megabytes in size.

This new resource is going live tomorrow on May 24th on TQ, and is currently live on Singularity. We are already working on some performance improvements to CREST and this resource, and are hoping that we can eventually increase the orders per page from 10k to 30k.

One of those changes we are making to help reach the speed and size we want for this resource is removing all the checks we do that verify the response matches the mediatype for this resource. The reason for that is that CREST spends a large amount of time verifying that and on a resource this size it ends up taking far too long. What this means to developers is that while we are mostly confident in the response being what it should, there is a chance that attributes might go missing or be of a type they shouldn't be with no warning. To be clear, this removal of response verification is only happening to this specific resource.

If you have any questions or concerns, please let us know. Hopefully this new resource helps elevate some of the work you are all having to do and makes life easier.

CCP FoxFour

]]>
New CREST Resource for Bulk Market Orders new-crest-resource-for-bulk-market-orders
http://developers.eveonline.com/blog/article/technical-note-integer-sizes-and-the-xml-api/ Technical note: Integer sizes and the XML API 2016-05-02T14:57:48Z 2016-05-02T14:57:48Z CCP Bartender 95465499 Hi all,

Just dropping a quick note on integer sizes with regard to the XML API access masks. The addition of the clones endpoint and it's associated access mask to the XML API has increased the maximum possible value of the API access mask from 2,147,483,647 to 4,294,967,295. These values are significant. When using a 32 bit integer to store a value, there are two possible ranges that the integer can be set to.

An unsigned int32 cannot be used to represent negative numbers, and has a range of values (0->4,294,967,295).

A signed int32 can be used to represent negative numbers, and although the magnitude of it's range is the same, it's range is evenly distributed between positive and negative values, and is thus (-2,147,483,647->2,147,483,647).

Until now, this hasn't mattered for the XML API. The maximum possible access mask value has always been lower than or equal to the maximum possible positive value of either type, so you could use either one. However, the maximum possible value is now larger than a signed int32 can hold. If you are using signed int32's in your application, it may error on some access masks. Please note that because the XML API does not use negative numbers as access masks, it is best practice at this time to use a 64 bit unsigned integer (range: 0->18,446,744,073,709,551,615)  or larger for access masks, as we may add more access masks in the future.

Carry on!

CCP Bartender

]]>
Technical note: Integer sizes and the XML API technical-note-integer-sizes-and-the-xml-api
http://developers.eveonline.com/blog/article/the-end-of-public-crest-as-we-know-it/ The end of public CREST as we know it 2016-04-18T14:19:14Z 2016-04-18T14:19:14Z CCP FoxFour 92168909 Hello everyone,

Really short dev blog to give you all a heads up on a change coming with the Citadel release in how we handle CREST.

We have made a fundemental change in CREST and no longer support public CREST. Instead of having public CREST for public resources and authed CREST for authenticated stuff we are now just going to have CREST on the crest-tq.eveonline.com domain. If you request a resource from CREST that should be public it will just work and won't require any authentication. Wonderful stuff this is! However we are unable to support public CREST when this change is deployed. We will add a 301 redirect from public-crest.eveonline.com to crest-tq.eveonline.com but if your client doesn't respect 301 redirects you may experience issues.

This change will be going live on April 27th during the downtime for the deployment of the Citadel release to TQ.

CCP FoxFour

]]>
The end of public CREST as we know it the-end-of-public-crest-as-we-know-it
http://developers.eveonline.com/blog/article/return-of-the-aggregated-character-stats/ Return of the aggregated character stats 2016-03-16T16:50:52Z 2016-03-16T16:50:52Z CCP SnowedIn 95538921 Greetings Space Friends,

 CCP SnowedIn on behalf of Team Tech Co here with good news about the aggregate character stats endpoint that some of you have built some amazing websites with in the past. As you may know when we had initially released it last year (powering fantastic apps such as http://spreadsheetsin.space/ by Bellatroix, https://youreveyear.appspot.com by Aaeriele and https://pozniak.pl/yearly-stats by Lukas Rox), it was pulling data from Singularity and was accessible via:

https://api-sisi.testeveonline.com/characters/<characterID>/statistics/year/<year>/

Well, as of today, March 16th, 2016, the endpoint is back. It is based off of Tranquility, and it's available at: 

https://characterstats.tech.ccp.is/v1/<characterID>/

You no longer need to request multiple times for multiple years, just once per characterID and you will get all of their aggregate years in the response. The response format a JSON dictionary with three keys, "aggregateYears", "characterID" and "characterName". The aggregateYears key contains keys of each year the player has aggregate stats for and each year should contain the same set of keys with values of how many times each action was performed by the character in the given year.

Authentication is handled via the characterStatsRead scope. Please ensure your application has access to use this scope in https://developers.eveonline.com/applications/. If you used the old endpoint in the past, your application will still need to be updated to make use of the new scope.

The data behind characterstats.tech.ccp.is is not live data. Instead it will be routinely updated, but the process is not something that we are willing to commit any sort of schedule to.

 Please let us know if you run into any problems using the new endpoint. We're excited and looking forward to see what you make with it!

 As an aside for anyone who's interested, this is now the second endpoint we have running on our Kubernetes infrastructure on Google's Cloud Platform with https://focusgrouplogs.tech.ccp.is being the first.

 

Happy coding!

]]>
Return of the aggregated character stats return-of-the-aggregated-character-stats
http://developers.eveonline.com/blog/article/the-eve-online-api-challenge/ The EVE Online API Challenge 2016-02-15T14:34:28Z 2016-02-15T14:34:28Z CCP FoxFour 92168909 The EVE Online API Challenge

For a long time, EVE Online has offered the ability for third-party developers to pull information from our database servers for use in other applications and websites. Over the years, this ability has spawned the growth of a substantial and active third-party developer community within the greater EVE Online community. As developers, we are constantly amazed by the things you create. We’ve seen everything: from incredibly comprehensive industry and mapping tools; mind boggling espionage and counter-espionage tools; to expansive killboards and so much more.

Over the years our offerings to third-party developers have gotten more substantial and our commitment to providing those services has increased as well. This year alone has seen incredible strides forward in what CREST offers with the addition of saved fits, contacts, and much more.

What Does CREST Offer

Here is a brief rundown of some of the things that CREST currently offers:

  • Read and write of character contacts
  • Read and write of saved fittings
  • Read a characters current location
  • Dogma information (all dogma attributes and dogma effects)
  • The ability to set the clients autopilot waypoint
  • Alliance information
  • Inventory information (categories, groups, and types including a types dogma information)
  • Universe information including regions, constellations, and solar systems
  • Tournaments
  • Market information
  • Sovereignty information
  • Wars
  • Incursions
  • Industry
  • Killmails

Continuing our efforts to support third-party developers, we are incredibly happy to announce The EVE Online API challenge: Create a piece of software that primarily focuses on use of the CREST API.

The goal of this challenge is to have some fun and strengthen the third-party developer community. We highly encourage those of you participating to think outside the box and do something new or different. Once all the entries are submitted and CCP has reviewed and judged them, we will share the results in another dev blog and possibly discuss them at Fanfest 2016. In addition and to help add to the fun of participating we will be giving away some prizes. But before we get to that, here are the rules you should be aware of.

Submitting An Entry

To submit an entry to the contest, follow this link and fill out the form. Please be aware that the character name(s) you enter will be used to contact winners, so please be sure they are accurate.

Judging

As part of the submission, all entries must provide a public Git repository link that should provide sufficient documentation, including a demo if at all possible. We'll be using the following criteria to determine the winners of the challenge:

  • Technical difficulty
  • Creativity/Originality
  • Clarity/Presentation
  • General polish

And Your Rewards Are...

Prizes will be awarded for 1st, 2nd and 3rd place entries. There will also be an additional prize awarded at the sole discretion of CCP Seagull.  (CCP Seagull was an early champion of CREST and helped lead the team that delivered it. Due to CCP Seagull's history with CREST, she has been given the honor of selecting one additional entry to win a prize completely at her own discretion.)

Prizes include two of the amazing TQ server blades that our operations team had put together. You can read a bit more about them here: https://community.eveonline.com/news/dev-blogs/fanfest-2015-silent-auction-servers-t-shirts-guns..-oh-my/

Alongside the server blades, we have a collection of SKINs, PLEX, Aurum, and signed books to give away. Each team member from the winning entry (maximum of two (2) team members per entry) will receive:

1st place:

  • Classic Tranquility server blade
  • Mordus Legion Parade SKIN Pack (Barghest, Orthrus, Garmur)
  • 5,000 Aurum
  • 5x PLEX

2nd place:

3rd place:

CCP Seagull's choice:

General Rules

  • The work submitted to this contest must be in English.
  • The contest starts on February 15th, 2016 (the moment this dev blog is published).
  • The deadline for all entries is March 4th, 2016 at 23:59 GMT.
  • Winners will be announced by March 29th 23:59 GMT.
  • You can work alone or with a partner, but teams cannot be larger than two (2) individuals.
  • The work submitted to this contest must be originally made for this contest. No pre-existing work is allowed and will be disqualified.
  • Work must be submitted with a link to a public Git repository.
  • Each team member must have a valid EVE Online account and character.
  • CCP is the sole judge of the contest and its judgment will be final.
  • Winners will be contacted via the email address associated with their character's EVE Online account.
  • While the focus of the challenge is on the use of the CREST API, use of other data sources (the XML API, SDE, or third-party) in support of the application is allowed.
  • Current partners, contractors, and employees of CCP Games are not eligible for participation in this contest.
  • You must be 18 years old or older to win.
  • Each person may only enter the contest one (1) time, whether as an individual or as a member of a team.
  • No purchase is necessary to enter or win.
  • CCP is not responsible for lost, late, misidentified or misdirected entries, or computer performance, errors, delays or failures.
  • If, for any reason, the fairness or integrity of the contest is questioned, or CCP's administration or fulfillment of the contest becomes impracticable, CCP reserves the right to terminate or modify the contest, and to disqualify any individual who tampers with the registration process or the administration of the contest.
  • Any and all disputes arising out of or in connection with this contest will be handled on an individual basis and in accordance with the laws of Iceland.
  • Contest void where prohibited or restricted by law, and all applicable federal, state and local laws and regulations apply.
  • By entering a submission, you grant CCP the right to use your name for advertising and marketing purposes, without additional compensation, unless otherwise prohibited by law.
  • Prizes are subject to change and will be awarded at CCP’s sole discretion.
  • Prizes may be taxable depending on your local laws.  All taxes are your responsibility.
  • All entries must be submitted under the MIT License and should include the following:

    Copyright © 2016 <your name>

    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

We are very excited to see what you all create for this. While working be sure to check out this auto-generated CREST documentation (http://jimpurbrick.com/crestmatic/) or the community written documentation over here (https://eveonline-third-party-documentation.readthedocs.org/en/latest/).

CCP FoxFour

(Cross-posted on the Community site)

FAQ

Q) Can I continue working on the application after submitting it to be judged?

A) Yes of course! However, since we will be judging the repo you submit through the Google Doc form, you should not push any changes to it after the deadline or you could disqualify yourself. Feel free to have the link you send us be a fork from your main repo where you continue working though.

Q) Can I participate if I don't have an active subscription?

A) Yes you can! If you wish to integrate the SSO or any part of authenticated CREST you will need to register an application on developers.eveonline.com which does have the requirement that you have paid for something from us in the past.

]]>
The EVE Online API Challenge the-eve-online-api-challenge
http://developers.eveonline.com/blog/article/crest-incursion-update-out-on-singularity/ Crest Incursion Update out on Singularity 2016-01-25T15:13:56Z 2016-01-25T15:13:56Z CCP TomB 153225188 Oi! Oi! Oi! We‘ve updated the data for CREST Incursions (which is now live on SiSi) to show you all the information you should ever need!

The type can now be retrieved through "incursionType", which currently has:

  1. "Incursion", which is used for the old school Sansha‘s Incursions and takes over a constellation
  2. "Spreading Incursion", which is used for the new Drifter Incursions and expands to new solar systems over time (ignoring constellation bounderies)

This should support any future type as well.

The actual list of solar systems can be retrieved through "infestedSolarSystems", which will return a list of each solar systems that is an active part of the infestation. This should now be used to indicate the affected solar systems instead of previously where the constellation has been used to get the systems. Please note that "Spreading Incursions" grow over time, so this data should be fetched on a regular basis.

And last but not least, the attacking force can now be seen through "aggressorFactionID"!

Let us know if you‘ve got issues, comments or just feel like sharing your love!

Much thanks from your friend,

CCP TomB

]]>
Crest Incursion Update out on Singularity crest-incursion-update-out-on-singularity
http://developers.eveonline.com/blog/article/crest-updates-for-january-2016/ CREST Updates For January 2016 2016-01-06T16:45:57Z 2016-01-06T16:45:57Z CCP FoxFour 92168909 Hello and welcome to a new year! Hopefully you all had a wonderful holidays and are enjoying the new year. I was fortunate enough that during the last few weeks the office was fairly empty around here and I was able to buckle down and get a large amount of work finished for CREST. Most of the following features were already a work in progress and just needed me to actually sit down to polish them off and test them. The following are also all available right now on Singularity and should be shipped to Tranquility on Tuesday the 12th of January 2016 with the YC 118.1 release. Without further rambling let's dig into the details of what is coming.

Alliance Resources

The alliance resources were a little bit of a mess. They were one of the first resources I ever wrote and contained a few glaring mistakes. First and foremost was this odd nesting of hrefs in hrefs. Here is how an item in the v1 of the Alliances collection looked:

{
    "href": {
        "id_str": "386292982",
        "shortName": "-10.0",
        "href": "https://public-crest.eveonline.com/alliances/386292982/",
        "id": 386292982,
        "name": "Pandemic Legion"
    }
},

That href inside of a href was just... wrong. Fixing this means making a backwards incompatible change so I created a v2 of the Alliances collection and deprecated v1. Be sure to read https://eveonline-third-party-documentation.readthedocs.org/en/latest/crest/versioning/ for more information on versioning. Essentially though if you are not sending an Accept header you are instantly going to get v2 when this ships to Tranquility  - which will probably break your stuff. If you're using v1 and sending an Accept header the response is going to contain an X-Deprecated header and your application should alert you to this.

The v1 resource will be removed later this year.

Items in v2 look something like this:

{
    "id_str": "99000004",
    "shortName": "ONE-A",
    "href": "http://crest.regner.dev/alliances/99000004/",
    "id": 99000004,
    "name": "One One Corporation Alliance"
}

I also fixed a defect where if an alliance executor corporation didn't exist, which is apparently possible, the Alliance resource would fail to parse. To fix that I have made the executorCorporation field optional. If you were getting 503 errors on some alliances in the past try them again after this release.

Universe Resources

Some fairly straight forward changes here. It used to be that to get all the solar systems in the universe you would have to start with the Regions collection, then get every region which would link to constellations, then get every constellation which would link to solar systems, and then get every solar system. This was usually OK because, if you wanted universe data, you usually wanted all of it. However this isn't always the case so I have added a Constellations collection and a Systems collection. Each is linked from the root of CREST and contains all of the regions or solar systems. The other nice advantage to this is it paves the way to having search for these resources since search should go on the collection.

Along with these changes I added an ID field to regions in the Regions collection and individual Region resource. The same was done for constellations and solar systems. So you should have no problem getting region, constellation, or solar system ID numbers without having to do some odd string parsing.

Personal Contacts

The ability to read a characters contacts from CREST is actually live on TQ right now. Funny that. There is a slight problem however in that if you have an alliance in your contacts it breaks. So this release fixes that problem. Along with that however I have taken it a step further and have added the ability to ...  write personal contacts. YAY!

Here is a sample of how the Contacts resource looks:

{
  "totalCount_str": "2",
  "items": [
    {
      "standing": -10,
      "alliance": {
        "id_str": "99000003",
        "href": "http://crest.regner.dev/alliances/99000003/",
        "id": 99000003,
        "name": "One One Corporation Alliance"
      },
      "href": "http://crest.regner.dev/characters/90000001/contacts/99000003/",
      "contact": {
        "id_str": "99000003",
        "href": "http://crest.regner.dev/alliances/99000003/",
        "name": "One One Corporation Alliance",
        "id": 99000003
      },
      "contactType": "Alliance"
    },
    {
      "standing": 0,
      "character": {
        "name": "Two One",
        "corporation": {
          "name": "Two One Corporation",
          "isNPC": false,
          "href": "http://crest.regner.dev/corporations/98000002/",
          "id_str": "98000002",
          "logo": {
            "32x32": {
              "href": "https://imageserver.eveonline.com/Corporation/98000002_32.png"
            },
            "64x64": {
              "href": "https://imageserver.eveonline.com/Corporation/98000002_64.png"
            },
            "128x128": {
              "href": "https://imageserver.eveonline.com/Corporation/98000002_128.png"
            },
            "256x256": {
              "href": "https://imageserver.eveonline.com/Corporation/98000002_256.png"
            }
          },
          "id": 98000002
        },
        "isNPC": false,
        "href": "http://crest.regner.dev/characters/90000002/",
        "capsuleer": {
          "href": "http://crest.regner.dev/characters/90000002/capsuleer/"
        },
        "portrait": {
          "32x32": {
            "href": "https://imageserver.eveonline.com/Character/90000002_32.jpg"
          },
          "64x64": {
            "href": "https://imageserver.eveonline.com/Character/90000002_64.jpg"
          },
          "128x128": {
            "href": "https://imageserver.eveonline.com/Character/90000002_128.jpg"
          },
          "256x256": {
            "href": "https://imageserver.eveonline.com/Character/90000002_256.jpg"
          }
        },
        "id": 90000002,
        "id_str": "90000002"
      },
      "contact": {
        "id_str": "90000002",
        "href": "http://crest.regner.dev/characters/90000002/",
        "name": "Two One",
        "id": 90000002
      },
      "href": "http://crest.regner.dev/characters/90000001/contacts/90000002/",
      "contactType": "Character",
      "watched": false,
      "blocked": false
    }
  ],
  "pageCount": 1,
  "pageCount_str": "1",
  "totalCount": 2
}

I even made sure there was an alliance in there as proof that I fixed that. That is a GET request for contacts but how about a POST request:

{
  "standing": 10,
  "contactType": "Alliance"
  "contact": {
    "id_str": "99000003",
    "href": "http://crest.regner.dev/alliances/99000003/",
    "name": "One One Corporation Alliance",
    "id": 99000003
  },
}

Both the GET and the POST request should be sent to the Contacts collection resource (/characters/<characterID>/contacts/). What if you want to edit or delete an existing contact. Then you can do a PUT or DELETE request to the Contact resource (/characters/<characterID>/contacts/<contactID/). PUT requests match the POST requests just as PUT requests instead and to a different resource. DELETE requests are well... empty DELETE requests to the appropriate Contact resource.

Contacts will have a characterContactsRead and a characterContactsWrite scope.

Character Location

If you take a look at the Character resource you will find a fancy new href pointing to a CharacterLocation resource. This resource is a simple GET resource that returns the current solar system of a character with a 10 second cache. Here is a sample of how it looks:

{
  "solarSystem": {
    "id_str": "30000140",
    "href": "http://crest.regner.dev/solarsystems/30000140/",
    "id": 30000140,
    "name": "Maurasi"
  }
}

Very simple and straight forward. The entire solarSystem element is optional. Basically if the character is offline you will get a blank dictionary back. I hope to add more information to this in the future such as ship type or station if they are in one. For now however this is the basics of it.

While we have put a fair bit of effort into figuring out how to do this efficiently on our end the question does come up as to how do you know when to start and stop polling this resource. At this point in time we don't have a way of notifying you that a character has come online and you should start polling. So my recommendation is that if you're a wormhole mapping site you start polling it when a character visits your service and then stop when you get an empty dictionary back. Maybe with a few extra pokes to make sure it wasn't a quick disconnect or something. Would love to hear what solutions you all come up with though.

Market IDs

Continuing the addition of IDs to resources the market types collection has an added ID field for the type ID and the market groups collection has an added ID field for the market group ID. Simple but hopefully helpful.

Dogma Information

The most important thing to be aware of about this next section is that if information is missing or presented in an unclear way please give me a shout and I will do my best to update it. There are three parts to this addition. The first the the addition of the DogmaAttributes collection which contains a list of all Dogma attributes. The second is the DogmaEffects collection which, you guessed it, contains a list of all Dogma effects. The third and final piece is all types now have a dictionary element called dogma added to them that contains two other optional elements. The first is effects and the second is attributes. Fairly straight forward stuff.

The one kicker to this information to be aware of is the Dogma effects. The DogmaEffect resource for individual effects contains three elements of unique importance. the preExpression, postExpression, and modifierInfo. If I understand what others have told me modifierInfo is actually a new way for us to do the pre and post expressions. All new dogma effects created use the newer modifierInfo way of storing this data. At some point in the coming months there is a plan to change all existing Dogma effects over to this modifierInfo format. At that point I may very well remove pre and post expression from this resource. Just something to be aware of going forward.

Here is a Dogma expression with the modifierInfo:

{
  "effectCategory_str": "0",
  "postExpression_str": "17934",
  "isAssistance": false,
  "description": "",
  "isOffensive": false,
  "disallowAutoRepeat": false,
  "isWarpSafe": false,
  "published": false,
  "electronicChance": false,
  "rangeChance": false,
  "effectCategory": 0,
  "id_str": "6395",
  "postExpression": 17934,
  "modifierInfo": [
    {
      "domain": "shipID",
      "modifiedAttributeID": {
        "id_str": "767",
        "href": "http://public-crest-sisi.testeveonline.com/dogma/attributes/767/",
        "id": 767,
        "name": "trackingSpeedBonus"
      },
      ...
  ],
  "preExpression": 17933,
  "displayName": "",
  "preExpression_str": "17933",
  "id": 6395,
  "name": "shipBonusEwWeaponDisruptionStrengthAC1"
}

You can read more about Dogma, and also a bit about the new and old way of doing effects, here: https://eveonline-third-party-documentation.readthedocs.org/en/latest/dogma/overview/

Types

As explained in the Dogma information section above the Type resource has had a large amount of Dogma information added to them. While touching the types however I also added an ID element to the response. Should make life for those of you working with the SDE and CREST a little easier.

Here is an example of how it looks:

{
  "capacity": 0,
  "description": "Utilizes a combination of containment field emitters and redundancy systems to prevent critical system damage. \r\n\r\nGrants a bonus to resistance for shield, armor and hull.\r\n\r\nOnly one Damage Control can be activated at a given time.",
  "portionSize_str": "1",
  "iconID": 77,
  "portionSize": 1,
  "iconID_str": "77",
  "volume": 5,
  "dogma": {
    "attributes": [
      {
        "attribute": {
          "id_str": "6",
          "href": "http://public-crest-sisi.testeveonline.com/dogma/attributes/6/",
          "id": 6,
          "name": "capacitorNeed"
        },
        "value": 1
      },
      ...
    ],
    "effects": [
      {
        "effect": {
          "id_str": "11",
          "href": "http://public-crest-sisi.testeveonline.com/dogma/effects/11/",
          "id": 11,
          "name": "loPower"
        },
        "isDefault": false
      },
      ...
    ]
  },
  "radius": 1,
  "id_str": "521",
  "published": true,
  "mass": 5000,
  "id": 521,
  "name": "Basic Damage Control"
}

Autopilot Waypoints

This should be a really fun one for web sites such as Dotlan or wormhole mappers. You can now send a POST request to CREST and have an autopilot destination set on the client.

If this is done however it will cause the autopilot to turn off if it is turned on for the client. There are a few reasons for this, one of which being a pilot not getting autopiloted to a low security system such as Tama or Amamake without their knowledge. To do this send a POST request to the Waypoints resource which is linked from the Character resource. The POST request should look something like the following:

{
    "solarSystem": {"href": "http://crest.regner.dev/solarsystems/30000142/", "id": 30000142},
    "first": false,
    "clearOtherWaypoints": true
}

The solarSystem element is pretty straight forward, it is the solar system you would like to be added to the waypoints of the client. For now only solar systems are supported. I admittedly completely forgot that you could set a waypoint to stations while writing this resource and need to go back and add support for that. The first element just specifies if the waypoint should be added to the front of the waypoint list as the first waypoint are the back of it as the last waypoint. clearOtherWaypoints also does exactly what it sounds like it should do. If set to true it clears all other waypoints from the waypoint list on the client.

Shoutout to Capt Out

A fine man by the name of Capt Out recently released a new blog on his site which you can read over here. Generally what he has to say is fairly relevant and awesome. In this case he has gone ahead and started generating documentation from the OPTIONS calls that CREST returns. You can see the documentation that gets generated here and the code behind generating it here.

Conclusion

Have fun! I am pretty sure this is the single largest update to CREST ever which makes for great excitement. Hopefully there will be lots more to come as well. At some point in the next few weeks I will be writing a dev blog for the community web site much like how I did when we released the saved fits in CREST. One of the things I did in that was include links to web pages or tools that had implemented the fittings already. If you do something with any of these changes, especially the Dogma info/waypoints/contacts/location, please let me know. You can reach me via EVE Mail on CCP FoxFour, the comments of this thread, or the Tweetfleet Slack.

Take care and dev on!

CCP FoxFour

]]>
CREST Updates For January 2016 crest-updates-for-january-2016
http://developers.eveonline.com/blog/article/read-write-of-saved-fits-now-on-sisi/ Read/Write of saved fits now on Sisi! 2015-10-14T13:20:58Z 2015-10-14T13:20:58Z CCP FoxFour 92168909 HOLY BATMAN IT'S HAPPENING! I think... assuming this testing goes well it will happen.

Right now you can go and create a CREST application for Sisi and request the scopes for fitting (characterFittingRead and characterFittingWrite). Upon requesting those scopes you can begin having people log into your site/application with those scopes at which point you should have authorization tokens that work for reading and writing saved fits. Here is how you go about using them.

If you follow CREST links from the root to the fittings resource you should go along the path of root -> decode -> character -> fittings.

Doing a GET on the character resource should get you results similar to the following:

{
  "totalCount_str": "1",
  "items": [
    {
      "description": "THIS IS A TEST FIT DESCRIPTION!",
      "fittingID_str": "258",
      "items": [
        {
          "flag": 11,
          "quantity_str": "1",
          "type": {
            "id_str": "518",
            "href": "http://crest.regner.dev/types/518/",
            "id": 518,
            "name": "Basic Gyrostabilizer"
          },
          "flag_str": "11",
          "quantity": 1
        }
      ],
      "fittingID": 258,
      "href": "http://crest.regner.dev/characters/90000001/fittings/258/",
      "ship": {
        "id_str": "587",
        "href": "http://crest.regner.dev/types/587/",
        "id": 587,
        "name": "Rifter"
      },
      "name": "THIS IS A TEST FIT NAME!"
    },
  ],
  "pageCount": 1,
  "pageCount_str": "1",
  "totalCount": 1
}

You can then do a POST to the fittings resource for that character that looks something like this to create a new fit:

{
  "name": "Name here...",
  "description": "Description here...",
  "items": [{
    "type": {
        "id": 518,
        "name": "Basic Gyrostabilizer",
        "href": "http://crest.regner.dev/types/518/"
    },
    "flag": 11,
    "quantity": 1
  }],
  "ship": {
    "id": 587,
    "name": "Rifter",
    "href": "http://crest.regner.dev/types/374/"
  }
}

You can then do a DELETE to a given fit resource to delete it. For a DELETE you don't need to send any data in the request, just do a DELETE request to the individual fit resource and you should be set.

The biggest disadvantage with this currently is that fits are cached for 15 minutes. So you could do a GET request but you won't see any changes from POST or DELETE requests for 15 minutes.

You can expect to get the following errors back from CREST:

  • FittingsStopSpamming
  • OwnerMaxFittings
  • FittingNameTooLong
  • FittingDescriptionTooLong

The fitting service has its own rate limit outside of the CREST rate limit and that can be hit easily if multiple applications are hitting the fitting resource, including the client.

Please let us know if you come across any issues with this feature. I would also like to request that you test this with the client open as you should see changes there instantly.

CCP FoxFour

]]>
Read/Write of saved fits now on Sisi! read-write-of-saved-fits-now-on-sisi
http://developers.eveonline.com/blog/article/a-note-on-crest-rate-limits/ A note on CREST rate limits 2015-10-13T21:18:09Z 2015-10-13T21:18:09Z CCP FoxFour 92168909 Hey people,

Just a heads up. The rate limit for public CREST was previously increased, and all the market details are available there. If you are currently scraping market details from authed CREST I would ask that you start doing it from public CREST instead. Public CREST should be faster and fail less often due to how we load balance the requests. Additionally, authed CREST will have its rate limit reduced in the next few weeks.

Public CREST rate limit: 150 requests a second

Authed CREST rate limit (to be implemented in a few weeks): 20 requests a second

Any questions, let us know,

CCP FoxFour

]]>
A note on CREST rate limits a-note-on-crest-rate-limits
http://developers.eveonline.com/blog/article/cache-timer-for-skilltree-reduced-1/ Cache Timer for SkillTree Endpoint Reduced 2015-08-18T16:48:48Z 2015-08-18T16:48:48Z CCP Tellus 95633963 The cache timer for the eve/SkillTree.xml.aspx endpoint in the XML API has been reduced to 24 hours from the previous duration of 10 years. That's all.

~~ CCP Tellus

]]>
Cache Timer for SkillTree Endpoint Reduced cache-timer-for-skilltree-reduced-1
http://developers.eveonline.com/blog/article/chat-channel-info-available-in-xml-api/ Chat Channel Information Now Available in XML API 2015-08-07T00:00:00Z 2015-08-07T00:00:00Z CCP Tellus 95633963 A new endpoint to the XML API is deployed on Tranquility.

Chat Channels

List of player chat channels you have created or are an operator of are exposed through the new endpoint char/ChatChannels.xml.aspx. It includes a list of characters, corporations, and alliances that have been allowed, blocked, muted, or been made an operator of the respective chat channel.

New access masks are required for access to this endpoint. None of your existing API keys will have access to this endpoint without first updating them and explicitly granting access to chat channels. The new access mask is 536870912.

Example XML

<?xml version='1.0' encoding='UTF-8'?>
<eveapi version="2">
  <currentTime>2015-08-07 14:48:08</currentTime>
  <result>
    <rowset name="channels" key="channelID" columns="ownerID,ownerName,displayName,comparisonKey,hasPassword,motd">
      <row channelID="-69329950" ownerID="95578451" ownerName="CCP Tellus" displayName="Players' Haven" comparisonKey="players'haven" hasPassword="False" motd="<b>Feed pineapples to the cats!</b>">
        <rowset name="allowed" key="accessorID" columns="accessorName">
          <row accessorID="99005629" accessorName="Tellus Alliance" />
        </rowset>
        <rowset name="blocked" key="accessorID" columns="accessorName,untilWhen,reason">
          <row accessorID="98396389" accessorName="Tellus Corporation" untilWhen="0001-01-01 00:00:00" reason="" />
        </rowset>
        <rowset name="muted" key="accessorID" columns="accessorName,untilWhen,reason">
          <row accessorID="90006031" accessorName="CCP Nestor" untilWhen="2015-08-07 15:17:40" reason="Test success! You can't speak now." />
        </rowset>
        <rowset name="operators" key="accessorID" columns="accessorName">
          <row accessorID="92168909" accessorName="CCP FoxFour" />
          <row accessorID="95465499" accessorName="CCP Bartender" />
        </rowset>
      </row>
    </rowset>
  </result>
  <cachedUntil>2015-08-07 15:45:08</cachedUntil>
</eveapi>

~~ CCP Tellus

]]>
Chat Channel Info available in XML API chat-channel-info-available-in-xml-api
http://developers.eveonline.com/blog/article/image-server-cdn-changes/ Image Server CDN changes 2015-07-21T15:29:02Z 2015-07-21T15:29:02Z CCP FoxFour 92168909 Hey people,

Just a quick heads up about a change with the image server. We've previously had a CDN in front of the image server (http://images.cdn1.eveonline.com/), but it never supported HTTPS. Due to this we previously recommended that people use the origin image server (https://image.eveonline.com/) as it supports HTTPS.

We are currently in the process of changing CDN providers, and one of the results of this change is that we have a new CDN in front of the image server that supports HTTPS. We are therefore now recommending that everyone switch their image server links to use the following address: https://imageserver.eveonline.com/

If you encounter any problems with this please let us know.

CCP FoxFour

]]>
Image Server CDN changes image-server-cdn-changes
http://developers.eveonline.com/blog/article/contact-labels/ Contact Labels Now Available via API 2015-07-13T14:51:55Z 2015-07-13T14:51:55Z CCP Tellus 95633963 Along with the Aegis Sovereignty release, the char/ContactList.xml.aspx and corp/ContactList.xml.aspx endpoints on the XML API have been updated to include your contact labels.

The rowsets "contactLabels", "corporateContactLabels", and "allianceContactLabels" have been added, each with the columns "labelID" and "name". The column "labelMask" has been added to the existing rowsets. LabelMask is a bitmask of the labelIDs indicating which labels have been assigned to each respective contact, or zero if the contact has no labels.

Example XML

<?xml version='1.0' encoding='UTF-8'?>
<eveapi version="2">
  <currentTime>2015-07-13 12:23:12</currentTime>
  <result>
    <rowset name="contactList" key="contactID" columns="contactID,contactName,standing,contactTypeID,labelMask,inWatchlist">
      <row contactID="90000002" contactName="Hemisphere" standing="10" contactTypeID="1385" labelMask="67108867" inWatchlist="True" />
      <row contactID="90000004" contactName="Comet" standing="5" contactTypeID="1379" labelMask="2097153" inWatchlist="True" />
    </rowset>
    <rowset name="contactLabels" key="labelID" columns="name">
      <row labelID="1" name="Alts" />
      <row labelID="2" name="Evil Twins" />
      <row labelID="4" name="Label 1" />
      <row labelID="8" name="Label 2" />
      <row labelID="16" name="aoeu" />
      <row labelID="2097152" name="Label 20" />
      <row labelID="67108864" name="Label 25" />
      <row labelID="2199023255552" name="Label 40" />
      <row labelID="36028797018963968" name="Label 54" />
      <row labelID="4611686018427387904" name="Label 61" />
    </rowset>
    <rowset name="corporateContactList" key="contactID" columns="contactID,contactName,standing,contactTypeID,labelMask">
      <row contactID="90000002" contactName="Hemisphere" standing="-5" contactTypeID="1385" labelMask="1" />
    </rowset>
    <rowset name="corporateContactLabels" key="labelID" columns="name">
      <row labelID="1" name="Corp Spies!" />
    </rowset>
    <rowset name="allianceContactList" key="contactID" columns="contactID,contactName,standing,contactTypeID,labelMask">
      <row contactID="90000002" contactName="Hemisphere" standing="5" contactTypeID="1385" labelMask="0" />
      <row contactID="90000003" contactName="Cosmos" standing="0" contactTypeID="1376" labelMask="1" />
      <row contactID="90000004" contactName="Comet" standing="-10" contactTypeID="1379" labelMask="3" />
    </rowset>
    <rowset name="allianceContactLabels" key="labelID" columns="name">
      <row labelID="1" name="Alliance Friend" />
      <row labelID="2" name="Stupid" />
    </rowset>
  </result>
  <cachedUntil>2015-07-13 12:25:12</cachedUntil>
</eveapi>

~~ CCP Tellus

]]>
Contact Labels contact-labels
http://developers.eveonline.com/blog/article/adding-bookmarks-to-the-api/ Adding Bookmarks to the API 2015-07-13T14:44:44Z 2015-07-13T14:44:44Z CCP Tellus 95633963 Dear fans of bookmarks, I bring good tidings.

A new endpoint to the XML API is to be deployed with the Aegis Sovereignty release. So without further ado, here are the details.

Bookmarks!

Your personal and corporation bookmarks are exposed through the new endpoints char/Bookmarks.xml.aspx and corp/Bookmarks.xml.aspx respectively. Information such as name, location, creator, and creation time are exposed. For your own security and safety in space, new access masks are required for access to these endpoints. None of your existing API keys will have access to these endpoints without first updating them and explicitly granting access to bookmarks.

The new access masks are 268435456 for character and account API keys, and 67108864 for corporation keys.

Very important disclaimer! Due to the sensitive nature of this endpoint, CCP reserves the right to remove or modify this endpoint at any time, at its own discretion.

Example XML

<?xml version='1.0' encoding='UTF-8'?>
<eveapi version="2">
  <currentTime>2015-07-08 21:42:12</currentTime>
  <result>
    <rowset name="folders" key="folderID" columns="folderName,creatorID">
      <row folderID="0" folderName="" creatorID="0">
        <rowset name="bookmarks" key="bookmarkID" columns="creatorID,created,itemID,typeID,locationID,x,y,z,memo,note">
          <row bookmarkID="12" creatorID="90000001" created="2015-07-08 21:34:14" itemID="60014689" typeID="57" locationID="30004971" x="0" y="0" z="0" memo="Home Station" note="Our base of residence" />
          <row bookmarkID="13" creatorID="90000001" created="2015-07-08 21:35:07" itemID="40314792" typeID="8" locationID="30004971" x="0" y="0" z="0" memo="Sun" note="" />
        </rowset>
      </row>
      <row folderID="1" folderName="A lovely empty folder" creatorID="90000001">
        <rowset name="bookmarks" key="bookmarkID" columns="creatorID,created,itemID,typeID,locationID,x,y,z,memo,note" />
      </row>
      <row folderID="3" folderName="Sites" creatorID="90000001">
        <rowset name="bookmarks" key="bookmarkID" columns="creatorID,created,itemID,typeID,locationID,x,y,z,memo,note">
          <row bookmarkID="16" creatorID="90000001" created="2015-07-08 21:37:12" itemID="40314827" typeID="15" locationID="30004971" x="0" y="0" z="0" memo="Duripant VII - Asteroid Belt 2 ( Asteroid Belt )" note="" />
          <row bookmarkID="17" creatorID="90000001" created="2015-07-08 21:37:22" itemID="40314829" typeID="15" locationID="30004971" x="0" y="0" z="0" memo="Duripant VII - Asteroid Belt 3 ( Asteroid Belt )" note="" />
          <row bookmarkID="18" creatorID="90000001" created="2015-07-08 21:37:29" itemID="40314794" typeID="15" locationID="30004971" x="0" y="0" z="0" memo="Duripant I - Asteroid Belt 1 ( Asteroid Belt )" note="" />
          <row bookmarkID="19" creatorID="90000001" created="2015-07-08 21:37:39" itemID="40314811" typeID="15" locationID="30004971" x="0" y="0" z="0" memo="Duripant VII - Asteroid Belt 1 ( Asteroid Belt )" note="" />
        </rowset>
      </row>
      <row folderID="4" folderName="Random crap" creatorID="90000001">
        <rowset name="bookmarks" key="bookmarkID" columns="creatorID,created,itemID,typeID,locationID,x,y,z,memo,note">
          <row bookmarkID="14" creatorID="90000001" created="2015-07-08 21:36:08" itemID="0" typeID="5" locationID="30004971" x="-373405654941.733" y="42718621667.0746" z="-1415023302173.46" memo="spot in Duripant solar system" note="" />
          <row bookmarkID="15" creatorID="90000001" created="2015-07-08 21:36:46" itemID="0" typeID="5" locationID="30004971" x="-373405652840.03" y="42718623812.4957" z="-1415023308332.07" memo="spot in Duripant solar system" note="" />
        </rowset>
      </row>
    </rowset>
  </result>
  <cachedUntil>2015-07-08 22:42:12</cachedUntil>
</eveapi>

~~ CCP Tellus

]]>
Adding Bookmarks to the API adding-bookmarks-to-the-api
http://developers.eveonline.com/blog/article/alliance-contract-item-fix/ Alliance Contract Items Fix 2015-07-10T11:01:50Z 2015-07-10T11:01:50Z CCP Tellus 95633963 Good news everyone!

We have deployed to Tranquility a fix for a long-standing bug in the XML API where alliance contracts were showing up in corp/Contracts.xml.aspx, but were not queryable via corp/ContractItems.xml.aspx.

With this fix deployed, you will now be able to query alliance contracts via corp/ContractItems.xml.aspx!

~~ CCP Tellus

]]>
Alliance Contract Item Fix alliance-contract-item-fix
http://developers.eveonline.com/blog/article/coordinates-in-outpost-list/ Coordinates in Outpost List 2015-07-08T15:03:54Z 2015-07-08T15:03:54Z CCP Tellus 95633963 Hey guys, we've got another spicy little change has been deployed to the XML API on Tranquility.

The eve/ConquerableStationList.xml.aspx and corp/OutpostList.xml.aspx endpoints have had three new attributes added to them: x, y, and z. These are the in-space coordinates of the respective stations.

Here's an example output from corp/OutpostList.xml.aspx:

<?xml version='1.0' encoding='UTF-8'?>
<eveapi version="2">
  <currentTime>2015-07-08 14:45:21</currentTime>
  <result>
    <rowset name="corporationStarbases" key="stationID" columns="stationID,ownerID,stationName,solarSystemID,dockingCostPerShipVolume,officeRentalCost,stationTypeID,reprocessingEfficiency,reprocessingStationTake,standingOwnerID,x,y,z">
      <row stationID="60014875" ownerID="98000001" stationName="H74-B0 III - Moon 4 - Aoeu" solarSystemID="30004034" dockingCostPerShipVolume="0" officeRentalCost="10000" stationTypeID="12242" reprocessingEfficiency="0.5" reprocessingStationTake="0.025" standingOwnerID="98000001" x="61066567680" y="2632949760" z="285129646080" />
    </rowset>
  </result>
  <cachedUntil>2015-07-08 14:46:21</cachedUntil>
</eveapi>

Your imaginary friend,
CCP Tellus

]]>
Coordinates in Outpost List coordinates-in-outpost-list
http://developers.eveonline.com/blog/article/aegis-sovereignty-api-changes/ Aegis Sovereignty API Changes 2015-07-07T11:05:06Z 2015-07-07T11:05:06Z CCP FoxFour 92168909 With just under a week to go before the new Sovereignty system hits Tranquility it's probably time that we start getting you information on the APIs we will be deploying to support it. So without further rambling, here’s the details.

Structures

All of the new APIs are in CREST and the first resource to look at is the SovStructures resource. This lists every structure in New Eden that is contributing to sovereignty be it a TCU, IHub, or a station. This is completely public information that will be available from within the client and so we are also making available via public CREST to everyone. Here is how it looks in the client:

Via the client you can show info on any solar system in the game to get details about the structures in that system. You can also find a list of all your alliances structures via the Corporation -> Alliances -> Sovereignty interface. Here is an example of how it will look in the CREST API: https://gist.github.com/Regner/78ad881108dcf2ea3600

Let’s walk through some of the details:

  • The route is: /sovereignty/structures/
  • We are going to start with a 5 minute cache on this resource.
  • This is a collection, which means it will have an items element containing a list of all the "things".
  • Collections may be paginated, but ideally they won’t be. I am waiting for this to hit Duality to get a better feel for how many structures there should be per page. Either way, you should be prepared to follow page links if they are present. See the alliances resource for an example.
  • Each entry in the items list contains:
    • solarSystem: The system in which the structure is located.
    • structureID: Unique item ID for this structure.
    • type: A reference to the type of structure this is. This contains its typeID, name, and a link to the types resource.
    • alliance: The alliance that owns the structure.
    • vulnerableStartTime: The next time at which the structure will become vulnerable. Or the start time of the current window if current time is between this and vulnerableEndTime.
    • vulnerableEndTime: The time at which the next or current vulnerability window ends. At the end of a vulnerability window the next window is recalculated and locked in along with the vulnerabilityOccupancyLevel. If the structure is not in 100% entosis control of the defender, it will go in to 'overtime' and stay vulnerable for as long as that situation persists. Only once the defenders have 100% entosis control and has the vulnerableEndTime passed does the vulnerability interval expire and a new one is calculated.
    • vulnerabilityOccupancyLevel: The occupancy level for the next or current vulnerability window. This takes into account all development indexes and capital system bonuses. Also known as Activity Defense Multiplier from in the client.  It increases the time that attackers must spend using their entosis links on the structure.

The trickiest part of that resource is understanding the vulnerability window information. The most important bit to understand is that at the end of a vulnerability window we calculate the next vulnerability window. It is at that time we look at the development indexes and capital system, and lock that information in. This is then used to calculate not only how big the vulnerability window will be but also what vulnerabilityOccupancyLevel bonus will be used for modifying things like the entosis link time.

You can find an example of this resource on Gist: https://gist.github.com/Regner/78ad881108dcf2ea3600

Campaigns

When a sovereignty structure is successfully contested by using entosis links, it will trigger what is called a Command Node Campaign - the event in which alliances must capture nodes throughout the constellation. Depending on a few conditions the type of campaign will be different. Before we dive into the structure of the API I just want to briefly review what that flow looks like.

For TCUs and IHubs: if the attackers manage to complete an entosis capture on the structure, it will create a structure defense campaign of type 1 or 2 respectively. These events pit the owning alliance against all other alliances in New Eden for the campaign. If the defender successfully wins the campaign, then the structure will revert back to its normal, owned state. However if they lose the campaign then the structure will explode. 

Stations are slightly different. They start off the same as a TCU or IHub, in that when the attackers complete the entosis capture a Station Defense campaign (type 3) is started, where the defending alliance is against all other alliances. The difference is that if the defenders lose, the station does not blow up like a TCU or IHub. Instead, when Station Defense campaign is won by the attackers the station goes into a Freeport mode, and schedules another campaign (type 4 - Station Freeport). This second campaign is different from the first 3 in that there is no defending alliance. All attackers show up in a list and simply the first to 100% wins.

Details for the campaigns resource:

  • The route is: /sovereignty/campaigns/
  • To start with it will have a 30 second cache time.
  • Like the structures resource it is a collection which can be paginated. It will start by listing all campaigns on one page but that may be changed.
  • Each entry in the items list contains:
    • campaignID: Unique ID for this campaign.
    • sourceItemID: The structure item ID that is related to this campaign.
    • sourceSolarsystem: The solar system the structure is located in.
    • constellation: The constellation in which the campaign will take place.
    • eventType: Type of event this campaign is for. 1 = TCU defense, 2 = IHub defense, 3 = Station defense, 4 = Station Freeport.
    • startTime: Time the event is scheduled to start spawning command nodes.
    • scores: This is an optional field that is present in eventType 4.
      • team: Alliance participating in the event.
      • score: Score for the specified alliance.
    • defender: This is an optional field that is present in eventType 1, 2, and 3.
    • defender: Defending alliance.
    • score: Score for the defending alliance.
    • Score for all attacking parties.
    • attackers: This is an optional field that is present in eventType 1, 2, and 3.

You can find an example of this resource on Gist: https://gist.github.com/Regner/69190d239609fb8b016a

Alliances

We have added two new optional fields to the alliances resource. Those fields are the primeHour and capitalSystem.

Test Server

These changes are available on Duality for testing right now. You can access it here: https://public-crest-duality.testeveonline.com

As always with a test server this information is subject to change.

]]>
Aegis Sovereignty API changes aegis-sovereignty-api-changes
http://developers.eveonline.com/blog/article/minor-change-to-locations-xml-aspx/ Minor change to Locations.xml.aspx 2015-07-02T15:18:32Z 2015-07-02T15:18:32Z CCP Tellus 95633963 We have deployed a small change to the Locations.xml.aspx endpoint to Tranquility.

It will no longer give a 135 error when some of the itemIDs being queried are invalid or not owned by the keyholder, instead those itemIDs are simply not included in the resulting rowset. This also implies that you may very well get an empty rowset now.

This change applies to both the character and corporation variants of this endpoint.

Have a nice day, courtesy of CCP Tellus!

]]>
Minor change to Locations.xml.aspx minor-change-to-locations-xml-aspx
http://developers.eveonline.com/blog/article/characteraffiliations-and-the-image-server/ CharacterAffiliations and the Image Server 2015-05-15T15:45:46Z 2015-05-15T15:45:46Z CCP FoxFour 92168909 Hello devs,

Writing a quick blog to update you on two small changes that have been made. The first change is to the eve/CharacterAffiliations endpoint for the XML API. For those that are unaware this resource lets you feed in a comma separated list of character ids and get back basic information such as name, corp, alliance, and faction. See an example here. Up until recently the resource cached each characters information separately for 1 hour and then no matter what any of the cache times actually were it would say the page was cached for 1 hour. This has been changed to only cache for 15 minutes and return a cachedUntil value that is calculated to be the longest time of all ids requested. We will be monitoring this change and acting accordingly but we hope to leave it at this new 15 minute cache time. This change is live on TQ now.

The second change is to the image server. We have added CORS headers to the image server itself. This was deployed to Singularity on Friday May 15th and if all goes well it will be deployed to Tranquility on Monday May 18th.

That's all for now. Happy coding. :)

CCP FoxFour

]]>
CharacterAffiliations and the Image Server characteraffiliations-and-the-image-server
http://developers.eveonline.com/blog/article/additions-to-the-image-export-collection/ Additions to the Image Export Collection 2015-05-05T15:03:26Z 2015-05-05T15:03:26Z CCP FoxFour 92168909 The Image Export Collection (IEC, https://developers.eveonline.com/resource/image-export-collection) contains a lot of really cool images and helps you people build amazing things. We love that and want to help encourage that. While at Fanfest this year, some of you had requests for things you would like to see added to the IEC. If you can think of anything else you would like to see added to the IEC be sure to let us know on the forums.

  • Tournament UI: used by the alliance tournament UI.
  • Crimewatch: stuff used for the criminal/suspect/limited engagement timers.
  • Skills: these are the skills icons for showing if you have the required skills or not. Primarily used on the market.
  • ShipUI: which is the hub with capacitor and module icons. This includes things such as overload, HP bars, the overheat image, and more.
  • InfoPanels: things from the info panels you see at the top left of the screen.
  • FlagIcons: the little icons placed on brackets or the overview for alliance, standings, etc.
  • Fitting: the basic UI elements for the fittings circle thing.
  • Opportunities: should be pretty straight forward.
  • SKINs: Images related to the new SKINs feature.
  • Hacking: images relating to the hacking mini-game. (Coming with the next release in June)

Again, if you have anything you want added that is not please let us know. The additions here are included in the most recent release of the IEC for Mosaic which can be found here: https://developers.eveonline.com/resource/image-export-collection.

One final note on the IEC before we move on: There was a change made in one of the recent releases as to how we name renders in the IEC. I guess there were cases where we would render an icon multiple times for different races and so the renders had a _raceID. This is no longer there. This change was made a while ago internally and only just got pushed live recently. The delay between the change being made and it going live caused us to fail to communicate this change outward. We are really sorry about the delay on telling you guys about this. You can see further discussion about it here: https://forums.eveonline.com/default.aspx?g=posts&m=5609292#post5609292

]]>
Additions to the Image Export Collection additions-to-the-image-export-collection
http://developers.eveonline.com/blog/article/ccp-games-on-github/ CCP Games on GitHub 2015-04-14T15:57:19Z 2015-04-14T15:57:19Z CCP FoxFour 92168909 Hello! I wanted to take some time to talk to you people a bit about GitHub and some of the areas we are using it within CCP to work with you.

There are more and more things showing up within the CCP Games GitHub organization, which can be found at https://github.com/ccpgames/. While things such as CCPWGL have been up for a while, there is not necessarily a push but a desire from many CCPers to put more things openly on GitHub. Our graphics engine team has already put their dae-to-red project along with their rescache project up there publicly. I know of a few other things that are likely to end up there as well.

While this is really awesome I would like to ask that if you submit a pull request or an issue to be both patient and helpful. Several of the people involved in these projects are very new to git and GitHub. Those of us with more experience will of course help them but I know from my own experience how helpful it is when the community pitches in even a little bit and has some patience. So if you do submit a pull request and it doesn't get any attention right away don't be afraid to give it a friendly bump or poke me to poke the maintainer.

License

Simple and to the point, we have decided to license our GitHub projects under the MIT license. We have added this license to all existing projects and will continue to add it to future ones.

CCPWGL

In a dramatic change from last years FanFest the CCP WebGL project received a big update on GitHub. It is in a pretty good working state right now with all the new models and other resources. It is our intention to keep updating it as needed and more closely watch pull requests. Since its update several issues have already been fixed and pull requests accepted. You can find the CCP WebGL project at https://github.com/ccpgames/ccpwgl.

Documentation

Documentation is something I have been thinking long and hard about. For the longest time CCP never had any official documentation for any of our third-party developer related resources. There are some OK wikis out there with good information but there has always been a desire to get some more official documentation from CCP out there. In that effort when we released the developers web site, https://developers.eveonline.com/ we tried putting some basic documentation up. From that experiment we have learned a few things. First, I am terrible at writing documentation and second that it takes a fair bit of time. The advantage of having CCP hosted documentation is it's generally a lot easier to find, because we control it we can link to it more officially. The disadvantage is that there's not as many resources that can be put into it.

While the developers site was a good experiment, some effort has been put into thinking of a new way that would combine the best of both worlds. One that that CCP controls and can safely link to, but also one that the community can contribute to. Bonus points for something that looks great, is easy to read, is easy to contribute to, and ideally is close to the development environment for third-party devs.

After doing some research into other possibilities and talking with the community some the current idea is to try out documentation written in markdown, hosted on GitHub, and rendered on Read The Docs. This has what I hope to be many advantages. First the generated docs look really good. You can see the generated output here: http://eveonline-third-party-documentation.readthedocs.org/en/latest/sso/intro/

Second because CCP maintains the GitHub repository for the documentation, we have the final say about what goes into it. Anyone can contribute however by forking the repository, committing some changes, and submitting a pull request. Since we maintain the repository, all those pull requests have to be approved by us. Continuing the reasons why GitHub seems like a great idea it's a place where a lot of third-party developers are already working. This means contributing to the documentation, and reading it, is generally very close to your current work.

If you take a look at the documentation and find a problem or that something is missing, please feel free to open an issue on the GitHub repo. That is one of the benefits for doing it this way.

You can find the GitHub repository for the documentation here: https://github.com/ccpgames/eveonline-third-party-documentation

That's all for now. Take care and fly safe!

CCP FoxFour

]]>
CCP Games on GitHub ccp-games-on-github
http://developers.eveonline.com/blog/article/a-quick-update-to-market-orders/ A quick update to market orders 2015-01-09T14:33:12Z 2015-01-09T14:33:12Z CCP FoxFour 92168909 Hope you all had a wonderful Christmas and New Years. This is just a very quick blog to let you guys know about a small change I was able to squeeze into Proteus upon returning from vacation. After allowing you guys to access real time market information, some of you asked for some extra information to be added to the results. Based on those requests we have added the order id, the id of the location, the id of the type, and the original order volume to the results.

Example entry from: https://api-sisi.testeveonline.com/market/10000002/orders/buy/?type=https://api-sisi.testeveonline.com/types/587/

{
    "totalCount_str": "20",
    "items": [
        {
            "volume_str": "332",
            "buy": true,
            "issued": "2014-10-31T16:21:23",
            "price": 223004.01,
            "volumeEntered": 444,
            "minVolume": 1,
            "volume": 332,
            "range": "station",
            "href": "https://api-sisi.testeveonline.com/market/10000002/orders/3725514252/",
            "duration_str": "90",
            "location": {
                "id_str": "60003760",
                "href": "https://api-sisi.testeveonline.com/universe/locations/60003760/",
                "id": 60003760,
                "name": "Jita IV - Moon 4 - Caldari Navy Assembly Plant"
            },
            "duration": 90,
            "minVolume_str": "1",
            "volumeEntered_str": "444",
            "type": {
                "id_str": "587",
                "href": "https://api-sisi.testeveonline.com/types/587/",
                "id": 587,
                "name": "Rifter"
            },
            "id": 3725514252,
            "id_str": "3725514252"
        }
    ],
    "pageCount": 1,
    "pageCount_str": "1",
    "totalCount": 20
}

The above is live on Sisi now and will ship with Proteus next week.

CCP FoxFour

]]>
A quick update to market orders a-quick-update-to-market-orders
http://developers.eveonline.com/blog/article/december-2014-third-party-developer-update/ December 2014 third-party developer update 2014-12-09T15:20:34Z 2014-12-09T15:20:34Z CCP FoxFour 92168909 char/CharacterSheet

Attribute Enhancers

Back in this dev blog we announced a change to the way the API returns implants. We also announced that the attributeEnhancers element was being deprecated  and would be removed come December. The actual message returned in the API said December 1st. The code has now been submitted to remove this element and will be deployed on December 9th with Rhea.

Clone Grades

First let’s talk about some small changes to the XML API for Rhea on December 9th. With the removal of clone grades in Rhea, the char/CharacterSheet endpoint contains a few elements that are now irrelevant. They are as follows:

<cloneTypeID>164</cloneTypeID>
<cloneName>Clone Grade Alpha</cloneName>
<cloneSkillPoints>0</cloneSkillPoints>

As with the change to implants in the character sheet in this dev blog, we will be assuming consumers of the API cannot handle elements they expect being there suddenly disapearing. For that reason the above is what will now be returned for all characters. We could have set cloneSkillPoints to something absurd such as 500m, but then people might not notice their application is still checking it. By setting cloneSkillPoints to 0 any application that checks it and compares to your current number of SP to see if the clone is out of date will always say you have an insufficient clone. On top of that, I am really looking forward to all instances of EVEMon alerting people that their clones are insufficient in Rhea. That will bring me great entertainment.

developers.testeveonline.com

An update to the developers site was required with the release of market data in CREST and the requirement for applications wishing to handle that data being authenticated. The new version of the devlopers site, which you are reading this blog on, lets you create the authentication only applications you can already create and also CREST applications. When you select “create a CREST application” you will get some new fancy options that let you select what scopes you would like your application to have access to.

For now the only available scope is publicData, but it's a start and the list will grow in time. So go now and create your publicData applications.

CREST

Public CREST SSL

So, public CREST. Let’s have a brief chat about it shall we? Right now it is available on both HTTP and HTTPS. I want to make this a formal notice that the HTTP, the non-SSL version, will be going away. Still figuring out the exact date for killing it off, but no sooner than March. If you have any objections to this idea please voice them now.

New Resources

Some new resources have been added and exposed to public CREST for information on the types in EVE. ItemCategories, ItemGroups, and ItemTypes are all available now from the root of the API for both public CREST and authed CREST. For now ItemType only includes the name and description of a type, but in time the hope is that other information such as Dogma will be included.

X-Deprecated

I want to stress once again the importance of watching for and adhering to the X-Deprecated header in CREST. If you are not sure what I mean about this please see this video here. The whole video is full of great information but I have linked to the section that talks about the X-Deprecated header.

In the next while some CREST resources will be marked for deprecation and it would be wonderful if your libraries and applications handled that gracefully. If you are looking for some resources to test your application or library on with the X-Deprecated header please see the application/vnd.ccp.eve.Api-v2+json representation of https://crest-tq.eveonline.com/

Static Data Export

The Static Data Export will receive a small update this release as well consisting of two small, but fairly important additions.

Those of you who work with the dogma data, especially fitting tools, you will be happy to know that the dogma expressions table is now included in the SDE. The exclusion of this table was mostly just an oversight, but has not been a huge issue as its use case is pretty limited and those of you who do use it got the data from the client cache. We really don't want you to have to do that though and so are working on getting it included in the SDE. You can find it now in the SDE as a table called dgmExpressions.

The second change is to the dogma effects (dgmEffects) table. There will be a new column called modiferInfo added. This is most important now for the new Amarr Tech 3 destroyer, the Confessor. This is a new way for us to author dogma on types and generally an improvement overall for us. You can expect to see more things using this method going forward. The kicker however is that a type may have a dogma effect that uses the older dogma expressions and new ones with the modiferInfo. If the effect has modifierInfo that takes precedence and the other information is ignored. Fun times ahead!

You can download the new version of the SDE, which includes the above changes, from this page: https://developers.eveonline.com/resource/static-data-export

SSO

Default Client Settings

When we launched the SSO publicly for everyone and let you go about creating your own clients for it we changed some default settings. Mainly this was setting it so that authentication only applications would not get refresh tokens and had an expiry time of 5 minutes. We are removing those changes so that authentication only applications and CREST applications will both get refresh tokens and have an expiry time of 20 minutes. The refresh tokens have an unlimited lifetime.

I feel I should also point out that there is a page on the community site that allows you to revoke an applications access: https://community.eveonline.com/support/third-party-applications/

We have heard from some people that users are often not sure what the SSO is and sometimes think websites are trying to steal their passwords. We are also working on a page to clearly explain to users what the SSO is, how to revoke access, and those kinds of things.

The changes to default client settings should hopefully be going live next week. I will make a quick post once they have gone live to let you all know.

Defining Who Can Sign-in

Right now if you use the SSO in your application people with banned or inactive accounts cannot sign-in. This is a pretty big disincentive to use the SSO. So we are changing it. Once the change goes live there will be no restriction on who can sign into your site via the SSO. On a related note we are looking into the restriction that you must have an active EVE account to create applications on the developers site. The restriction on having paid us money in the past will stay, but we want to remove the active account restriction there. Third-party developers do amazing things and we want to support you in every way possible.

As with the default client setting changes, we are not 100% positive on the deployment date so we will let you know once they are deployed.

Final Notes

There is a fair bit of stuff in this dev blog. As always I will be watching the comments and answering any questions I can. This will probably be the last dev blog this year though so I wanted to go over a few final things.

While things have been moving fairly fast the last 6 or 8 months of this year in terms of tools and resources for third-party developers things are going to slow down for a bit. First of course there is Christmas coming up and with it some vacation time. At the beginning of next year however I will also be focusing on some internal stuff. The NGINX configs for CREST and public CREST need to be completely rewritten along with some other infrastructure for CREST needing some updates. That work will probably take longer than I want but is required to help ensure the best performance of CREST and that we can continue marching forward. That's just the start though as there is a pile of other technical debt I want to take a chunk out of early next year. 

So while you will probably notice development of things for third-party developers slows a bit, I want you all to know it's for a good reason and in the long run will be really good for everyone.

That's it for now, thanks for reading space people.

CCP FoxFour

]]>
December 2014 third-party developer update december-2014-third-party-developer-update
http://developers.eveonline.com/blog/article/nom-nom-tastey-market-data/ Nom nom tasty market data! 2014-11-24T10:36:42Z 2014-11-24T10:36:42Z CCP FoxFour 92168909 Yay! More dev blogs for the developer's site. This time we are going to talk about market data.

To be clear upfront we do NOT believe this will, nor is this an attempt to, kill web sites like EVE Central. For one the format of this data is not very friendly for many applications. It is not the best optimized if you are trying to bulk get data for many different items from many different regions, and other such things. Web sites such as EVE Central and their APIs will be around for a long time to come. We are however fairly hopeful EVEMon can stop cache scraping after this.

Market Organization

When it comes to dealing with market data one of the most important things is to know is what types are on the market to even get data about. For that purpose we have the MarketTypes resource available at the following route: /market/types/

We also are making available the MarketType resource. MarketTypes is a collection of all possible MarketType resources. You can access individual MarketType resources with the following route: /market/types/<typeID:int>/

The MarketTypes resource is paginated with up to 1,000 results per page. With 11,273 types available on the market at the time of writing that results in 12 pages.

You can also filter the MarketTypes resource by specifying a group, which you can get from the MarketGroups resource found at the following route: /market/groups/

The MarketTypes resource takes a group parameter and expects it to be a href to a MarketGroup.

Example:

GET /market/types/?group=http://public-crest-sisi.testeveonline.com/market/groups/2/ HTTP/1.1
Host: public-crest-sisi.testeveonline.com
Accept: application/vnd.ccp.eve.MarketTypeCollection-v1+json

 

The Universe

Due to a technical decision in the early days of EVE the market data is split per region. Which means that you need region information in CREST if you want to be able to get all the market data from CREST without knowing about the regions from the SDE. For the reason we have gone and cleaned up some of the universe resources and gotten them ready to be opened up. You can now access the Regions, Region, Constellation, SolarSystem, and Planet resources.

Starting with Regions found at /regions/ you will get a list of all regions in the game. From that you can follow the links to an individual Region resource under /regions/<regionID:int>/ which will give you all the information about the region, including the constellations in that region. I think you can guess where this is going. From there you can follow the links to each constellation, then to each solar system, and finally to each planet.

This should give you an always accurate picture of the universe.

One of the key elements to this however is back in the Region resource. Each Region resource along with giving details on the region such as name, X, Y,Z, and the constellations also gives two other links to MarketBuyOrders and MarketSellOrders resources for that region.

Putting It Together

It's important to remember that one of the ideas behind CREST is that applications should not have to construct URIs themselves. For more information on that I highly recommend you check out this presentation here. So with that in mind, the first step is to get the URI to types you want market orders for. If you simply want all types on the market you would go from the root to MarketTypes. Each item in that resource will have a href for the type, get that.

Next assuming you want the market orders for every region in the game, you would go from the root to Regions, follow the href for each item to the individual Region resource, and get the MarketBuyOrders href.

With the href for each region's MarketBuyOrders and the href for all the types, you can send a GET request to the region's MarketBuyOrders with the added parameter of type that includes the href to that type.

Doing it the way described above, while specifying an Accept header, means that if we ever change the routes or format for any of these resources your application will keep on working, and I wouldn't mind moving all the universe data to /universe/regions/ instead of just /regions/.

Example:

GET /market/10000002/orders/buy/?type=https://api-sisi.testeveonline.com/types/683/ HTTP/1.1
Host: https://api-sisi.testeveonline.com
Authorization: Bearer jKVB8oaN9qboU5kQG4sWSoWxzSUaFkQaUyeisy8jWU3apRfYSgYsKpZGNbLh41xXEzuy-NDBX1FohEdEadaukQ2
Accept: application/vnd.ccp.eve.MarketOrderCollection-v1+json

 

Gaining Access

The resource for market groups and market types along with all the universe data will be exposed via public CREST. Actually, it's all on Sisi right now: public-crest-sisi.testeveonline.com

The market buy order and market sell order resources will NOT be made available on public CREST. There are several reasons for this but actually this is a good thing for you guys. Since this resource wont be in public CREST but we want you to access it we need to give you authed CREST. Authed CREST actually has a substantially higher rate limit as well. This will require creating an application here on the developers site and select the publicData scope. Yes, this does mean it's going to be basically impossible to get this data straight from CREST into things such as excel or Google Docs as you will have to send custom headers along with going through the whole OAuth flow. However doing this through authed CREST gives us more confidence in releasing resources, it preps the way for authed CREST to private data, and gives us better monitoring/control over people using it.

Work on upgrading the developers site to allow you to create applications that require scopes is currently ongoing and should be ready for Rhea. For now however as I am sure many of you want to test this stuff on Sisi make a post in this dev blogs comments thread and include your applications name and client ID. I will then update the application to receive refresh tokens along with having the appropriate scope. As this will be done manually please understand if it's a bit slow. :)

Conclusion

That's all for now. I do expect a site replicating the market UI in short order guys. You have the market history and market orders information now. I should point out that there is a discussion on going about making the market history resource work like the market orders resource with a parameter. We will keep you updated on how that goes.

CCP FoxFour

]]>
Nom nom tastey market data! nom-nom-tastey-market-data
http://developers.eveonline.com/blog/article/public-crest-rate-limit/ Public CREST Rate Limits 2014-11-18T14:15:33Z 2014-11-18T14:15:33Z CCP FoxFour 92168909 I began writing this intro with "Hey developers" and then my mind starting thinking about DEVELOPERS DEVELOPERS DEVELOPERS! After smashing my face into the desk in an attempt to forget that reference I was unable to come up with a new introduction.

Public CREST Caching

Before we talk about the rate limit on public CREST lets talk about some caching. I wrote a dev blog recently talking about some changes we made to the caching for public CREST. Both those changes were made to the NGINX config, one was simply bumping the cache size and the other was prep work for a change that went out in Phoebe.

Up until Phoebe we could only ever specify a cache time in seconds for CREST and this was done via a decorator in Python. This meant that all calls to that resource always got the same cache time. It meant that if we wanted to cache something like say the market history we would generally set its cache to 1 hour. The unfortunate thing is that the market history data doesn't change during the day. It only changes at midnight UTC. So the same data was often being refetched throughout the day because the 1 hour cache expired. We couldn't just set the cache to be 24 hours though because if someone grabbed something at 2AM for the first time, it wouldn't be available until 2AM the next day. That's no good.

So on the backend for CREST we added the ability for developers to pass functions into the cache decorator, and then when the resource was called we could calculate a time in seconds to cache. So for things like the market history this means that we can pass in a function like CalculateSecondsUntilMidnight, and no matter when during the day that resource is called it will cache until midnight.

These changes to the CREST backend were deployed with Phoebe and have made a huge difference in the number of cache hits we get which is always nice to see as a developer. And because I know how much you guys like graphs, here is a graph of the cache hits for all HTTP200 requests on public CREST:

The biggest change is in the Expire cache status. Those expires came mostly from the market history resource getting requested at 01:00, and then again much later in the day after the cache had expired. With the cache now lasting until midnight, that is much less an issue.

New Rate Limits

With the improved caching and continued monitoring of public CREST we decided to increase its rate limit. I have posted the new rate limits on the Public CREST page over on the Developers site as these changes are live on TQ now. For quick reference though they are now as follows:

  • Rate Per Second: 30
  • Burst Size: 100

Hope this makes your life a bit easier, better, and helps allow you guys do more cool stuff! :D

CCP FoxFour
Team Size Matters

p.s. that graph so didn't take 3 devs 30-40 minutes learning R to create... I swear.

]]>
Public CREST Rate Limits public-crest-rate-limit
http://developers.eveonline.com/blog/article/and-now-welcome-to-developers-eveonline-com/ And now welcome to developers.eveonline.com 2014-10-31T16:28:13Z 2014-10-31T16:28:13Z CCP FoxFour 92168909 Just over a month ago, in late September, we launched https://developers.testeveonline.com on Singularity. This was a huge step forward for us and something we have been trying to do for a while. Seeing it actually come to life was cause for great excitement here at CCP. Now that we have had some good testing on the site, which lead to a few changes and fixes, we are happy to move the site to production on Tranquility. Here is a brief overview of what has changed:

Required Forms of Payment

To log in and create applications on the developer site we require you to have paid us money at some point and have a validated email address. Up until now the only form of payment that was considered was credit card. When signing into the developer site for the first time we will now check if you have paid us with any of our currently valid payment methods. This should help a lot for those of you in countries where credit cards are not very common.

Developer License

The final version of the developer license agreement was published. You can view it here or when signing into the developer site for the first time. This does mean that to work with the SSO, and CREST in the future, you will have to agree to the license agreement.

SSO Applications for Tranquility

When you create an application on the developer site now it will be created for TQ and your applications will need to point to https://login.eveonline.com/ instead of https://sisilogin.testeveonline.com/.

SSO Examples

Since the SSO is now going live on TQ I thought it would be nice to point out a few examples of the SSO in use.

  • edk_eve_sso - A plugin for the EDK Killboard by former CSM representative Two step. Give users the ability to comment on kills with their EVE characters.
  • flask-eve-sso-example - A simple example integrating the EVE SSO with Flask using the Flask-OAuthlib library.
  • eve-sso - An example of using the EVE SSO on Windows Phone 8.1

Documentation

If you browse the resources section of the developer site you will notice that while it is a dramatic improvement over what we had before, it is still missing information and even has some sections that are completely empty. We have not forgotten about this and are still working on getting all those pages updated. We however have been spending a bit of time on a different bit of documentation.

Work has begun on creating a CREST resource that fully describes the entirety of CREST. When done this resource will be made available on both CREST and public CREST. You can see an example (please note changes may still occur) of what this will look like here.

RESTful CREST

If you were unable to attend EVE Vegas--which I highly recommend you do because it's awesome--you would have missed a great presentation by Capt Out. Capt Out is a former CCP employee who helped build CREST and has an incredible amount of knowledge about it. If you have the time I would seriously recommend viewing the presentation he did on YouTube here.

CCP + Open Source

We love open source stuff, we love seeing you guys open source your stuff, and we really love the general attitude of the third-party developer community of working together. With that in mind we have a few small things of note.

CCP Merc has begun work on a python library called crestClient and will push it to GitHub once it's closer to complete. I hope to contribute to it myself and hope that if anyone else out there has any suggestions on it contributes as well. You can watch the CCP Games GitHib to see when it is released, hopefully next week.

I have continued my efforts to contribute as well by committing and making pull requests to the python library for the XML API called evelink. The general idea being after we announce changes to the XML API I attempt to see if I can get them added to the library before the changes go live.

If you know of any other XML API or CREST libraries please consider adding them to XML API Libraries or CREST Libraires on EVElopedia.

That's all for now,

CCP FoxFour

]]>
And now welcome to developers.eveonline.com and-now-welcome-to-developers-eveonline-com
http://developers.eveonline.com/blog/article/more-additions-to-char-charactersheet/ More Phoebe changes to char/CharacterSheet 2014-10-29T16:25:39Z 2014-10-29T16:25:39Z CCP FoxFour 92168909 Following my previous dev blog, Jump clones, implants, skills, and more, we have made a few more minor additions to the char/CharacterSheet endpoint.

Jump Fatigue Information

With the addition of Jump Fatigue in Phoebe it seemed appropriate to include it in the API for players to track their own characters information, possibly so corporations can track their members information to plan operations and/or for character trading. Being able to confirm a characters jump fatigue before purchasing seemed like an important thing. Looking at you Chribba and eveboard.

<jumpActivation>2014-10-28 17:33:41</jumpActivation>
<jumpFatigue>2014-10-28 17:33:35</jumpFatigue>
<jumpLastUpdate>2014-10-28 14:47:01</jumpLastUpdate>

Both jumpFatigue and jumpActivation are time stamps for when those counters expire. So in the above example the character will be able to jump again at 17:33:41 on 2014-10-28 and their fatigue will expire at the same time. jumpLastUpdate is when the last jump occurred by that character. This was included for those who wish to show something like what the EVE client does with timers (such as these). To If you wanted to show something like that you need a start time, and thats what jumpLastUpdate is.

If you would like to know more about these changes you can view the original dev blog here.

Remotely Moving A Clone

With Phoebe there are also new restrictions on where you can set your medical clone to. You can again read more about it in the dev blog linked above, specifically under the "Does this make it too difficult for new pilots to get out into 0.0?" FAQ. To help players track when they can move their clone again we have added the information to the API.

<remoteStationDate>2014-10-28 14:50:56</remoteStationDate>

Both of the above changes are now live on Sisi. So feel free to test away and let us know of any issues.

Thats all for now,

CCP FoxFour

]]>
More Phoebe changes to char/CharacterSheet more-additions-to-char-charactersheet
http://developers.eveonline.com/blog/article/industry-changes-for-phoebe/ Industry Changes for Phoebe 2014-10-10T00:00:00Z 2014-10-10T00:00:00Z CCP Nullarbor 91303426 Hi Space Devs!

We have some changes coming in Phoebe for industry data. You should first make sure you are up to date with the player facing changes as detailed in CCP Ytterbiums developer blog Lighting The Invention Bulb. Then we can go through what impact this has for developers.

Static Data Export

To begin with, the static data export for blueprint data had some things moved around. Each activity type is now a string instead of a number so that we can produce a meaningful schema definition behind the scenes. The list of products, materials and skills have been turned into lists instead of yaml objects, for the same reasons.

The valid list of activity names are as follows:

  • manufacturing
  • copying
  • research_time
  • research_material
  • invention

You should notice that reverse engineering has been removed, and all ancient relics now have an invention activity. The consume flag which was present on some materials has also been removed, all materials are now consumed. You can see a complete example of the changes here:

1128:
  activities:
    copying:
      time: 1440
    invention:
      materials:
      - quantity: 2
        typeID: 20411
      - quantity: 2
        typeID: 20414
      products:
      - probability: 0.4
        quantity: 10
        typeID: 2025
      skills:
      - level: 1
        typeID: 11433
      - level: 1
        typeID: 11455
      - level: 1
        typeID: 23087
      time: 30900
    manufacturing:
      materials:
      - quantity: 1
        typeID: 40
      - quantity: 242
        typeID: 34
      - quantity: 148
        typeID: 35
      - quantity: 569
        typeID: 36
      - quantity: 1
        typeID: 39
      products:
      - quantity: 1
        typeID: 577
      skills:
      - level: 1
        typeID: 3380
      time: 1800
    research_material:
      time: 630
    research_time:
      time: 630
  blueprintTypeID: 1128
  maxProductionLimit: 100

XML API

  • /corp/IndustryJobsHistory.xml.aspx
  • /char/IndustryJobsHistory.xml.aspx
  • /corp/IndustryJobs.xml.aspx
  • /char/IndustryJobs.xml.aspx

The authenticated jobs APIs have a new field related to invention called successfulRuns which will indicate how many runs resulted in a product being delivered. This will only apply to invention jobs which can now have multiple runs submitted but only part of them succeed. All other job types this will just be equal to the number of runs for the job.

Similarly the status column from the jobs resource which used to contain a success or failure state will now just indicate that the job was delivered.

Removed

  • 104 = Succeeded
  • 105 = Failed

Added

  • 101 = Delivered

The successfulRuns field and the changes to status will be applied to all jobs returned from the API, including those that were delivered before Phoebe.

Phoebe SDE

There are a large number of industry related data changes for Phoebe as part of a complete invention rebalance. We have built a pre-release version of the SDE for you to have a look over. However keep in mind that this release is still not locked in stone, you can expect more data changes in the coming weeks.

http://cdn1.eveonline.com/data/Phoebe_0.9_106316_db.zip

We will provide a final updated SDE when the release hits on November 4.

That's it for now, thanks for being awesome and making great industry tools!

]]>
Industry Changes for Phoebe industry-changes-for-phoebe
http://developers.eveonline.com/blog/article/jump-clones-implants-skills-and-more/ Jump clones, implants, skills, and more 2014-10-09T00:00:00Z 2014-10-09T00:00:00Z CCP FoxFour 92168909 This is a fairly decent number of changes, one of which is backwards incompatible. Since these changes introduce backwards incompatability and potentially break some peoples applications we do ask that you let others know as much as possible and share this news around.

account/AccountStatus

Multiple Character Training Information

It is already possible to see the skill queue for multiple characters. EVE Mon shows me both characters from an account that are training for example. What EVE Mon and other applications cannot do right now is remind you when your multiple character training is about to expire, or more importantly when multiple characters are not training and could be. With that in mind we have gone ahead and added the multiple character training information to account/AccountStatus.

<rowset name="multiCharacterTraining" key="trainingEnd" columns="trainingEnd">
  <row trainingEnd="2014-12-11 14:15:16" />
  <row trainingEnd="2014-12-11 20:20:20" />
</rowset>

The key part to look at here is the multiCharacterTraining rowset. The number of rows in this rowset will indicate how many extra characters can be trained. This should only return multi character trainings that end at a date later than when called, nothing historical (caching aside of course).

char/CharacterSheet

Home Station

This is a really simple and straight forward change. We simply added a homeStationID element to the result element.

<homeStationID>60011566</homeStationID>

Simple but hopefully useful.

Respecs

There are three new elements relating character respecs, probably better known to you guys as neural remaps.

<lastRespecDate>0001-01-01 00:00:00</lastRespecDate>
<lastTimedRespec>0001-01-01 00:00:00</lastTimedRespec>
<freeRespecs>2</freeRespecs>

lastRespecDate is the date of the last respec including free ones while lastTimedRespec is the date of the last timed respec.

Unallocated Skill Points

<freeSkillPoints>50000</freeSkillPoints>

That should be pretty straight forward as it's just a simple integer with the number of unallocated skillpoints that a character has.

Jump Clones

This is the largest addition to this endpoint and includes several different bits. All of this is meant to allow you guys to track where jump clones are, what implants they have, and the time until you can clone jump again. Should hopefully be some pretty useful stuff.

The first thing is the jump clone date. This is the time the last jump was made, you then add the base time modified by skills.

<cloneJumpDate>2014-10-08 20:54:50</cloneJumpDate> 

Next up is the jump clones themselves and should be pretty self-explanatory:

<rowset name="jumpClones" key="jumpCloneID" columns="jumpCloneID,typeID,locationID,cloneName">
  <row jumpCloneID="2" typeID="164" locationID="60014930" cloneName="" />
  <row jumpCloneID="4" typeID="164" locationID="60014848" cloneName="herps derps" />
  <row jumpCloneID="5" typeID="164" locationID="60014842" cloneName="" />
</rowset>

cloneName is something new that CCP Karkur added for Phoebe. You can now right click on a jump clone in the client and set its name. That name will then also be exposed here. The location ID might be a bit tricky when it's that of a clone vat bay, not entirely sure how you as a developer will be able to look that up. Didn't however want that to stop us from getting it into the API now.

The final section for jump clones is the implants for them and that will look like this:

<rowset name="jumpCloneImplants" key="jumpCloneID" columns="jumpCloneID,typeID,typeName">
  <row jumpCloneID="4" typeID="20499" typeName="High-grade Slave Alpha" />
  <row jumpCloneID="4" typeID="20501" typeName="High-grade Slave Beta" />
  <row jumpCloneID="4" typeID="20503" typeName="High-grade Slave Delta" />
  <row jumpCloneID="4" typeID="20505" typeName="High-grade Slave Epsilon" />
  <row jumpCloneID="4" typeID="20507" typeName="High-grade Slave Gamma" />
  <row jumpCloneID="4" typeID="20509" typeName="High-grade Slave Omega" />
  <row jumpCloneID="4" typeID="33068" typeName="QA SpaceAnchor Implant" />
  <row jumpCloneID="5" typeID="22119" typeName="Mid-grade Slave Alpha" />
  <row jumpCloneID="5" typeID="22120" typeName="Mid-grade Slave Beta" />
  <row jumpCloneID="5" typeID="22121" typeName="Mid-grade Slave Delta" />
  <row jumpCloneID="5" typeID="22122" typeName="Mid-grade Slave Epsilon" />
  <row jumpCloneID="5" typeID="22123" typeName="Mid-grade Slave Gamma" />
  <row jumpCloneID="5" typeID="22124" typeName="Mid-grade Slave Omega" />
</rowset>

With that you should have all relevant information for jump clones. If we have missed anything please let us know and we will see about getting it added. :)

Character Implants

Above we talked about implants for jump clones, and some of you may have noticed that included more than just the attribute implants where as right now the character sheet only tells you about attribute implants for your active clone. We decided to add all implants to your current clone, but that required a new format for displaying them, new code for processing the information, and a new DB procedure for actually getting the implant information. The result of that is that the old way of displaying implants in the XML API is being deprecated.

Right now the information is contained in an element called attributeEnhancers. While that element will stay, there will not be anything in it; the same as if the character had no implants. Implants instead will be listed in a new rowset called implants. All together it will look something like this:

<attributeEnhancers>
  <deprecated explanation="This section of the character sheet has been deprecated. Please use the new implants section to get a characters current implants. Final removal of this section is scheduled for December 1st 2014. Thanks to kais58 and LuciaDenniard for pointing out how terrible I am at spelling." />
</attributeEnhancers>
<rowset name="implants" key="typeID" columns="typeID,typeName">
  <row typeID="33516" typeName="High-grade Ascendancy Alpha" />
  <row typeID="33525" typeName="High-grade Ascendancy Beta" />
  <row typeID="33528" typeName="High-grade Ascendancy Gamma" />
  <row typeID="33526" typeName="High-grade Ascendancy Delta" />
  <row typeID="33527" typeName="High-grade Ascendancy Epsilon" />
</rowset>

If all goes according to plan, that attributeEnhancers element will be removed come December.

Deployment

All of the changes described above are live on Sisi right now so you can go and test there. They will be deployed to TQ with Phoebe on November 4th. As some of you have been asking, yes once I am back from EVE Vegas I will also get the jump fatigue information into the XML API for Phoebe as well.

CCP FoxFour

]]>
Jump clones, implants, skills, and more jump-clones-implants-skills-and-more
http://developers.eveonline.com/blog/article/lowering-the-blueprints-cache-time/ Lowering the blueprints cache time 2014-10-03T00:00:00Z 2014-10-03T00:00:00Z CCP FoxFour 92168909 When we launched the blueprint endpoints back in August we set the cache time to 24 hours. This was due to how heavy of a resource those endpoints can be and we were concerned about performance. Now that the endpoints have been live for a while and we have been able to monitor their usage and impact we are ready to lower their cache time. For now that means going from 24 hours to 12 hours. We will continue to monitor the situation and make changes.

This change is on Sisi now and will probably go to TQ on Monday October 6th.

As always, enjoy and let us know if there are any issues. :D

CCP FoxFour
@CCP_FoxFour

]]>
Lowering the blueprints cache time lowering-the-blueprints-cache-time
http://developers.eveonline.com/blog/article/a-note-about-caching-in-public-crest/ A note about caching in public CREST 2014-10-02T00:00:00Z 2014-10-02T00:00:00Z CCP FoxFour 92168909 We deployed two updates to public CREST today aimed at increasing our caching performance. The end result is that you should start seeing more cache hits on public CREST. Along with that however you will also notice a new header being passed. This header has been in CREST for some time, but was just added to public CREST with the cache changes.

You should now start seeing X-Cache-Status returned with all requests which will tell you if you got a cache hit, miss, or expire. For the nerdy who want to know what's going on behind the scenes: Up until recently the public CREST NGINX config was very straight forward. Your request came in, we did some finagling, and then that same route did the proxy pass upstream. We recently split this up a bit and the proxy pass is in a sub-request now. To get the cache status into our logs the sub-request adds the cache status to the X-Cache-Status header before returning the response. The initial route then takes the header value and stores it in a variable that we use in the logging.

The second change that we made today was to up the max size for the NGINX cache from 1g to 16g.

Now that the developer site is available with a dev blog channel just for third party developers I hope we can include more information like this. If you notice any issues please let us know, but hopefully everything should be good.

CCP FoxFour
@CCP_FoxFour

]]>
A note about caching in public CREST a-note-about-caching-in-public-crest
http://developers.eveonline.com/blog/article/welcome-to-developers.testeveonline.com/ Welcome to developers.testeveonline.com 2014-09-23T00:00:00Z 2014-09-23T00:00:00Z CCP FoxFour 92168909 :D :D :D :D :D

:D :D :D :D :D

:D :D :D :D :D

:D :D :D :D :D

I really don't think there is a way to express the joy we have in releasing this website, although I did try above. This website represents CCP's continued belief in offering support to third-party developers who give so much value to our players through their efforts. Along with launching this site I'm happy to say I've been reassigned with the primary purpose of supporting third-party developers. That includes things such as writing documentation, fixing defects with the XML API, expanding CREST, and whatever else is deemed the right thing to do for third-party developers.

Documentation

One of the goals of this website is to offer a central location for official CCP documentation. While EVElopedia works, it is a lot harder to search, browse, and just in general find the correct information. So while EVElopedia will continue to be used for things such as Getting Started Examples, information on what the XML API is, what CREST is, and that kind of thing will be located on the developer website.

The documentation we are releasing now is very general documentation. As stated above, it includes answers to questions like “what does CCP offer” and “what is the XML API” as well as links to external resources. Documentation on specific endpoints for the XML API and CREST is something that we are working on getting out, but for now they still do not exist. This is still a giant step in the right direction.

If you browse the resources section you will probably notice that there are still bits and pieces missing and/or incomplete. One of the beautiful things about this site is that the content can be edited directly by dev teams, not just the web devs. So we can update all the documentation, including things like the link to the SDE. So please send suggestions for edits, tweaks, improvements to the documentation and I’ll try to shoot out changes quickly.

Expanding the EVE Online SSO

We recently began letting a small, but growing, number of websites access and use the EVE Single-Sign-On. That was the first step in a plan that we outlined at FanFest this year which has the final goal of getting authenticated read and write access to CREST out to third-party developers. The second step in this plan is allowing anyone who would like to develop with the SSO to do so without having to request access from me directly. Our goal with the plan outlined at FanFest was to take the very minimal steps required, to help ensure the plan actually happened. While looking at getting everyone access to the SSO we clearly needed some way to allow them to register. We could build a new website from scratch, or we could look at past projects that never got released. That look into the past is what brought this developers website back from the dead.

With the developer’s website now live comes an expansion of the SSO for third-party developers. You can now login, register as a third party developer, and begin creating applications. For now applications can be authentication only applications, ones that use the SSO and not CREST.

Please keep in mind, that while this site is on Sisi any applications you create are going to be for Sisi and not TQ. Once the website goes live on TQ you will have to recreate your applications over there. This should however let you get started with testing.

Third-Party Dev Blogs

One of the huge benefits with having a website dedicated to third-party developers is that it is the natural place for dedicated third party-developer dev blogs. Previous to this if we had an announcement to make but it wasn't really considered large enough for a full dev blog it would just be a post on the technology lab forums. This was primarily due to not wanting to spam the normal dev blog channel with things only relevant to third-party developers.

Now that we have a channel for dev blogs dedicated to third-party developers you can expect to see a lot more updates posted here--when we make minor changes to an API, deploy something for testing, request some feedback--this will be the main channel of third-party developercommunicatn. For that reason I highly recommend you subscribe to the RSS feed or just come back here often!

That's all for now. I hope you guys are as excited about this as we are here. Fly safe good pilots.

CCP FoxFour
@CCP_FoxFour

]]>
Welcome to developers.testeveonline.com welcome-to-developers.testeveonline.com