FAQ: What Happens When the Open or save Dialog is Cancelled within the Pythonaddins Module

Answer

The cancel operation returns the 'NoneType' object (None). This is similar to the boolean objects of True and False.
[O-Image]

In addition to the dialog function, there is the MessageBox function which has a variety of button options based upon its defined type. This function returns a string value representing the message button pressed (i.e., 'Cancel', 'Ignore', 'Continue').

Note:
An 'if' clause can be used to determine the next steps within the Python code after a button is clicked. For example:

Code:
import arcpy
import pythonaddins

class Open(object):
    """Implementation for Dialog_addin.OpenButton (Button)"""
    def __init__(self):
        self.enabled = True
        self.checked = False
    def onClick(self):
        #Define the map document and data frame
        mxd = arcpy.mapping.MapDocument("CURRENT")
        df = arcpy.mapping.ListDataFrames(mxd,"*")[0]

        #Call the open dialog
        openValue = pythonaddins.OpenDialog("Open Dialog", False, r"C:\temp", "Open")
        #Make sure dialog was not canceled.
        if openValue != None:
            #Create the layer 
            rasterLayer = arcpy.mapping.Layer(openValue)
            #Add the layer to the map document
            arcpy.mapping.AddLayer(df, rasterLayer,"TOP")
            # Refresh map document
            arcpy.RefreshActiveView()
            arcpy.RefreshTOC()
            del mxd, df, openValue
        else:
            pythonaddins.MessageBox("No data will be added to the map document.", "Warning Message", 0)
del mxd, df, openValue