uk.ac.liv.auction.agent
Class MDPStrategy

java.lang.Object
  uk.ac.liv.auction.agent.AbstractStrategy
      uk.ac.liv.auction.agent.FixedQuantityStrategyImpl
          uk.ac.liv.auction.agent.AdaptiveStrategyImpl
              uk.ac.liv.auction.agent.DiscreteLearnerStrategy
                  uk.ac.liv.auction.agent.MDPStrategy

All Implemented Interfaces:

AdaptiveStrategy, AuctionEventListener, java.lang.Cloneable, FixedQuantityStrategy, Parameterizable, Prototypeable, Resetable, java.io.Serializable, Strategy

public class MDPStrategy

extends DiscreteLearnerStrategy

implements java.io.Serializable

A trading strategy that uses an MDP learning algorithm, such as the Q-learning algorithm, to adapt its trading behaviour in successive auction rounds. The current market-quote is hashed to produce an integer state value.

Version:

$Revision: 1.18 $

Author:

Steve Phelps

See Also:

Serialized Form

Field Summary

protected double	askBinStart
protected double	askBinWidth
protected double	bidBinStart
protected double	bidBinWidth
protected boolean	firstShout
protected MDPLearner	learner
static java.lang.String	P_ASKBINSTART
static java.lang.String	P_ASKBINWIDTH
static java.lang.String	P_BIDBINSTART
static java.lang.String	P_BIDBINWIDTH
static java.lang.String	P_LEARNER
static java.lang.String	P_QUOTEBINS
protected int	quoteBins

Fields inherited from class uk.ac.liv.auction.agent.DiscreteLearnerStrategy

markupScale

Fields inherited from class uk.ac.liv.auction.agent.AbstractStrategy

agent, auction, currentShout

Constructor Summary

MDPStrategy()

MDPStrategy(AbstractTradingAgent agent, double askBinStart, double askBinWidth, double bidBinStart, double bidBinWidth)

Method Summary

int	act() Generate an action from the learning algorithm.
int	auctionState(Auction auction) Hash the market quote to produce a state value for the learning algorithm.
Learner	getLearner()
void	learn(Auction auction) Perform learning.
void	reset() Reinitialise our state to the original settings.
void	setLearner(Learner learner)
void	setup(ec.util.ParameterDatabase parameters, ec.util.Parameter base) Initialise this object from a parameter database.
java.lang.String	toString()

Methods inherited from class uk.ac.liv.auction.agent.DiscreteLearnerStrategy

endOfRound, getMarkupScale, initialise, modifyShout, setMarkupScale

Methods inherited from class uk.ac.liv.auction.agent.FixedQuantityStrategyImpl

determineQuantity, getQuantity, setQuantity

Methods inherited from class uk.ac.liv.auction.agent.AbstractStrategy

eventOccurred, getAgent, modifyShout, protoClone, setAgent

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface uk.ac.liv.auction.agent.Strategy

determineQuantity, modifyShout, setAgent

Methods inherited from interface uk.ac.liv.util.Prototypeable

protoClone

Methods inherited from interface uk.ac.liv.auction.core.AuctionEventListener

eventOccurred

Field Detail

learner

protected MDPLearner learner

bidBinStart

protected double bidBinStart

bidBinWidth

protected double bidBinWidth

askBinStart

protected double askBinStart

askBinWidth

protected double askBinWidth

quoteBins

protected int quoteBins

firstShout

protected boolean firstShout

P_LEARNER

public static final java.lang.String P_LEARNER

See Also:

Constant Field Values

P_QUOTEBINS

public static final java.lang.String P_QUOTEBINS

See Also:

Constant Field Values

P_ASKBINSTART

public static final java.lang.String P_ASKBINSTART

See Also:

Constant Field Values

P_ASKBINWIDTH

public static final java.lang.String P_ASKBINWIDTH

See Also:

Constant Field Values

P_BIDBINSTART

public static final java.lang.String P_BIDBINSTART

See Also:

Constant Field Values

P_BIDBINWIDTH

public static final java.lang.String P_BIDBINWIDTH

See Also:

Constant Field Values

Constructor Detail

MDPStrategy

public MDPStrategy(AbstractTradingAgent agent,
                   double askBinStart,
                   double askBinWidth,
                   double bidBinStart,
                   double bidBinWidth)

MDPStrategy

public MDPStrategy()

Method Detail

setup

public void setup(ec.util.ParameterDatabase parameters,
                  ec.util.Parameter base)

Description copied from interface: Parameterizable

Initialise this object from a parameter database.

Specified by:

setup in interface Parameterizable

Overrides:

setup in class DiscreteLearnerStrategy

act

public int act()

Description copied from class: DiscreteLearnerStrategy

Generate an action from the learning algorithm.

Specified by:

act in class DiscreteLearnerStrategy

learn

public void learn(Auction auction)

Description copied from class: DiscreteLearnerStrategy

Perform learning.

Specified by:

learn in class DiscreteLearnerStrategy

auctionState

public int auctionState(Auction auction)

Hash the market quote to produce a state value for the learning algorithm.

reset

public void reset()

Description copied from interface: Resetable

Reinitialise our state to the original settings.

Specified by:

reset in interface Resetable

Overrides:

reset in class AbstractStrategy

getLearner

public Learner getLearner()

Specified by:

getLearner in interface AdaptiveStrategy

setLearner

public void setLearner(Learner learner)

Specified by:

setLearner in interface AdaptiveStrategy

toString

public java.lang.String toString()